A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery
From MaRDI portal
Publication:820791
Abstract: This paper introduces a simple principle for robust high-dimensional statistical inference via an appropriate shrinkage on the data. This widens the scope of high-dimensional techniques, reducing the moment conditions from sub-exponential or sub-Gaussian distributions to merely bounded second or fourth moment. As an illustration of this principle, we focus on robust estimation of the low-rank matrix from the trace regression model . It encompasses four popular problems: sparse linear models, compressed sensing, matrix completion and multi-task regression. We propose to apply penalized least-squares approach to appropriately truncated or shrunk data. Under only bounded moment condition on the response, the proposed robust methodology yields an estimator that possesses the same statistical error rates as previous literature with sub-Gaussian errors. For sparse linear models and multi-tasking regression, we further allow the design to have only bounded fourth moment and obtain the same statistical rates, again, by appropriate shrinkage of the design matrix. As a byproduct, we give a robust covariance matrix estimator and establish its concentration inequality in terms of the spectral norm when the random samples have only bounded fourth moment. Extensive simulations have been carried out to support our theories.
The popular assumption of sub-Gaussian or subexponential noises in the theoretical analysis of standard statistical procedures is contradictory and has adverse impact on the popularly used methods. Hence, simple and effective principles for robust statistical inference are needed for dealing with heavy tailed data. The paper under review addresses this problem. Its authors propose truncation of univariate data and,more generally, shrinkage of multivariate data to achieve the robustness. The authors' approach is illustrated through a general model, which embraces the linear model, matrix compressed sensing, matrix completion, and multitask regression as specific examples. The authors develop the generalized loss, the truncated and shrinkage sample covariance, and corresponding M-estimators. Then, the conditions required on the robust covariance inputs to ensure the desired statistical error rates of the M-estimator are presented. These results are thereafter applied to all the specific above-mentioned problems in order to explicitly derive the specific error rates. Next, the convergence properties of the shrinkage covariance estimator under the spectral norm are studied. It is found that under high dimensions the proposed robust covariance is minimax optimal up to a logarithmic factor, whereas traditional sample covariance is not. Finally, simulation analyses is presented, which demonstrate the advantage of proposed robust estimators over the standard ones.
Recommendations
Cites work
- A simpler approach to matrix completion
- A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers
- Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising
- Adaptive Huber Regression
- Adaptive thresholding for sparse covariance matrix estimation
- Atomic decomposition by basis pursuit
- Challenging the empirical mean and empirical variance: a deviation study
- Compressed sensing
- Concentration inequalities and moment bounds for sample covariance operators
- Covariance regularization by thresholding
- Empirical risk minimization for heavy-tailed losses
- Estimation of (near) low-rank matrices with noise and high-dimensional scaling
- Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
- Estimation of high-dimensional low-rank matrices
- Exact matrix completion via convex optimization
- Geometric median and robust estimation in Banach spaces
- Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
- High-dimensional graphs and variable selection with the Lasso
- High-dimensional robust precision matrix estimation: cellwise corruption under \(\epsilon \)-contamination
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- Inference and uncertainty quantification for noisy matrix completion
- Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
- Large covariance estimation through elliptical factor models
- Loss minimization and parameter estimation with heavy tails
- Minimax Rates of Estimation for High-Dimensional Linear Regression Over \ell_q-Balls
- Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
- Nearly unbiased variable selection under minimax concave penalty
- Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization
- Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
- On the prediction loss of the Lasso in the partially labeled setting
- One-step sparse estimates in nonconcave penalized likelihood models
- Optimal rates of convergence for sparse covariance matrix estimation
- Reconstruction From Anisotropic Random Measurements
- Restricted strong convexity and weighted matrix completion: optimal bounds with noise
- Robust estimators in high-dimensions without the computational intractability
- ROP: matrix recovery via rank-one projections
- Simultaneous analysis of Lasso and Dantzig selector
- Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-Rank Matrices
- Sparsistency and rates of convergence in large covariance matrix estimation
- Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
- Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries
- Sure independence screening for ultrahigh dimensional feature space. With discussion and authors' reply
- The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
- The restricted isometry property and its implications for compressed sensing
- Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements
- Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eQTL mapping
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(52)- High-dimensional robust approximated M-estimators for mean regression with asymmetric data
- Robust parameter estimation of regression models under weakened moment assumptions
- High dimensional generalized linear models for temporal dependent data
- Robust covariance estimation for distributed principal component analysis
- Differential network inference via the fused D-trace loss with cross variables
- User-friendly covariance estimation for heavy-tailed distributions
- The study of robust matrix regression models and algorithms
- Robust Recommendation via Social Network Enhanced Matrix Completion
- A framework of regularized low-rank matrix models for regression and classification
- Adaptive robust large volatility matrix estimation based on high-frequency financial data
- Large volatility matrix analysis using global and national factor models
- Rate-optimal robust estimation of high-dimensional vector autoregressive models
- Robust matrix estimations meet Frank-Wolfe algorithm
- Robust inference for high‐dimensional single index models
- Covariance Estimation for Matrix-valued Data
- Mining the factor zoo: estimation of latent factor models with sufficient proxies
- Robust high-dimensional tuning free multiple testing
- Understanding Implicit Regularization in Over-Parameterized Single Index Model
- D4R: doubly robust reduced rank regression in high dimension
- Low-rank matrix recovery under heavy-tailed errors
- Robust signal dimension estimation via SURE
- ARFIS: an adaptive robust model for regression with heavy-tailed distribution
- How do noise tails impact on deep ReLU networks?
- Truthful and privacy-preserving generalized linear models
- Dynamic Matrix Recovery
- Semi-supervised inference for the high-dimensional quantile regression
- Introducing HYBRID and ENSEMBLE: novel nonconvex penalization strategies for robust variable selection under missing data
- Sparse linear regression when noises and covariates are heavy-tailed and contaminated by outliers
- Structured matrix learning under arbitrary entrywise dependence and estimation of Markov transition kernel
- Robust distributed precision matrix estimation for high-dimensional data
- Statistical inference for matrix-vector linear regression without debiasing under Kronecker covariance structure
- Robust estimation for dynamic spatial autoregression models with nearly optimal rates
- Robust feature screening via Grothendieck's correlation with FDR control
- Outlier robust and sparse estimation of linear regression coefficients
- High-Dimensional Multivariate Linear Regression with Weighted Nuclear Norm Regularization
- Fast and Robust Low-Rank Learning over Networks: A Decentralized Matrix Quantile Regression Approach
- Robust Matrix Completion with Heavy-Tailed Noise
- Residual permutation test for regression coefficient testing
- Regularized adaptive Huber matrix regression and distributed learning
- Robust multi-task regression with shifting low-rank patterns
- Computationally efficient and statistically optimal robust high-dimensional linear regression
- Multiple regression for matrix and vector predictors: models, theory, algorithms, and beyond
- Sparse robust discriminant analysis for high-dimensional and heavy-tailed data
- Sharp bounds for multiple models in matrix completion
- Tail-robust factor modelling of vector and tensor time series in high dimensions
- The adaptive robust and efficient variable selection method for the linear regression model
- Simultaneous estimation of change-points in epidemic change models
- Noise variance estimation in a large-dimensional approximate factor model and its application
- Distribution-free robust independence test for high-dimensional data via semi-Grothendieck's covariance
- Estimation of sparse linear regression coefficients under L-subexponential covariates
- High-Dimensional Spatial Autoregression with Latent Factors by Diversified Projections
- Robust variable selection procedure with range constraints on the coefficient
This page was built for publication: A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q820791)
