High dimensional robust M-estimation: asymptotic variance via approximate message passing

DOI10.1007/s00440-015-0675-zzbMath1357.62220arXiv1310.7320OpenAlexW2108394050WikidataQ59478906 ScholiaQ59478906MaRDI QIDQ343797

Andrea Montanari, David L. Donoho

Publication date: 29 November 2016

Published in: Probability Theory and Related Fields (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1310.7320

zbMATH Keywords

Gaussian noiseFisher informationasymptotic varianceapproximate message passingapproximate message passing (AMP)robust M-estimationrobust regression estimators

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Linear regression; mixed models (62J05) Robustness and adaptive procedures (parametric inference) (62F35) Random matrices (probabilistic aspects) (60B20) Analysis of variance and covariance (ANOVA) (62J10)

Related Items

Fundamental barriers to high-dimensional regression with convex penalties, Approximate message passing algorithms for rotationally invariant matrices, The main contributions of robust statistics to statistical science and a new challenge, Asymptotics for high dimensional regression \(M\)-estimates: fixed design results, SLOPE is adaptive to unknown sparsity and asymptotically minimax, Activation function design for deep networks: linearity and effective initialisation, The asymptotic distribution of the MLE in high-dimensional logistic models: arbitrary covariance, Learning curves of generic features maps for realistic datasets with a teacher-student model*, A precise high-dimensional asymptotic theory for boosting and minimum-\(\ell_1\)-norm interpolated classifiers, Automatic bias correction for testing in high‐dimensional linear models, Fundamental limits of weak recovery with applications to phase retrieval, Detangling robustness in high dimensions: composite versus model-averaged estimation, Confidence regions and minimax rates in outlier-robust estimation on the probability simplex, Debiasing convex regularized estimators and interval estimation in linear models, Approximate message passing for sparse matrices with application to the equilibria of large ecological Lotka-Volterra systems, A Friendly Tutorial on Mean-Field Spin Glass Techniques for Non-Physicists, Universality of approximate message passing with semirandom matrices, Noisy linear inverse problems under convex constraints: exact risk asymptotics in high dimensions, Universality of regularized regression estimators in high dimensions, Adaptive and robust multi-task learning, The Lasso with general Gaussian designs with applications to hypothesis testing, A tradeoff between false discovery and true positive proportions for sparse high-dimensional logistic regression, Moderate-Dimensional Inferences on Quadratic Functionals in Ordinary Least Squares, Which bridge estimator is the best for variable selection?, Asymptotic risk and phase transition of \(l_1\)-penalized robust estimator, Finite-sample analysis of \(M\)-estimators using self-concordance, Statistical mechanics of the inverse Ising problem and the optimal objective function, On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators, Overcoming the limitations of phase transition by higher order analysis of regularization techniques, Can we trust the bootstrap in high-dimension?, The likelihood ratio test in high-dimensional logistic regression is asymptotically a rescaled Chi-square, Distributed linear regression by averaging, Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach, Concentration Inequalities for Statistical Inference, High-dimensional linear models: a random matrix perspective, The distribution of the Lasso: uniform control over sparse balls and adaptive parameter tuning, Multidimensional linear functional estimation in sparse Gaussian models and robust estimation of the mean, The scaling limit of high-dimensional online independent component analysis, Entropy and mutual information in models of deep neural networks*, Precise statistical analysis of classification accuracies for adversarial training, Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices, Approximate survey propagation for statistical inference, Generalized approximate survey propagation for high-dimensional estimation ^*, A Unifying Tutorial on Approximate Message Passing, Asymptotic normality of robust \(M\)-estimators with convex penalty

Cites Work

• Unnamed Item
• Unnamed Item
• An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
• Construction of pure states in mean field models for spin glasses
• Finding hidden cliques of size \(\sqrt{N/e}\) in nearly linear time
• Robust regression: Asymptotics, conjectures and Monte Carlo
• Rigorous solution of the Gardner problem
• Universality in polytope phase transitions and message passing algorithms
• On robust regression with high-dimensional predictors
• Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing
• Information, Physics, and Computation
• An Introduction to Random Matrices
• One-Step Huber Estimates in the Linear Model
• State evolution for general approximate message passing algorithms, with applications to spatial coupling
• The LASSO Risk for Gaussian Matrices
• The Noise-Sensitivity Phase Transition in Compressed Sensing
• Applications of the Lindeberg Principle in Communications and Statistical Learning
• The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
• Robust Estimation of a Location Parameter
• Robust Statistics