A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery

DOI10.1214/20-AOS1980zbMATH Open1479.62034arXiv1603.08315OpenAlexW3192637965MaRDI QIDQ820791FDOQ820791

Authors: Ziwei Zhu, Jianqing Fan, Weichen Wang 

Publication date: 28 September 2021

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: This paper introduces a simple principle for robust high-dimensional statistical inference via an appropriate shrinkage on the data. This widens the scope of high-dimensional techniques, reducing the moment conditions from sub-exponential or sub-Gaussian distributions to merely bounded second or fourth moment. As an illustration of this principle, we focus on robust estimation of the low-rank matrix $Thet{a}^{*}$ from the trace regression model $Y=Tr(Thet{a}^{\ast T}X)+epsilon$. It encompasses four popular problems: sparse linear models, compressed sensing, matrix completion and multi-task regression. We propose to apply penalized least-squares approach to appropriately truncated or shrunk data. Under only bounded $2+delta$ moment condition on the response, the proposed robust methodology yields an estimator that possesses the same statistical error rates as previous literature with sub-Gaussian errors. For sparse linear models and multi-tasking regression, we further allow the design to have only bounded fourth moment and obtain the same statistical rates, again, by appropriate shrinkage of the design matrix. As a byproduct, we give a robust covariance matrix estimator and establish its concentration inequality in terms of the spectral norm when the random samples have only bounded fourth moment. Extensive simulations have been carried out to support our theories.

Full work available at URL: https://arxiv.org/abs/1603.08315

Recommendations

• Robust sparse covariance estimation by thresholding Tyler's M-estimator
• Robust low-rank matrix estimation
• Rank penalized estimators for high-dimensional matrices
• Robust matrix completion
• ROP: matrix recovery via rank-one projections

zbMATH Keywords

shrinkagehigh-dimensional statisticsrobust statisticslow-rank matrix recoverytrace regressionheavy-tailed data

Mathematics Subject Classification ID

Asymptotic properties of nonparametric inference (62G20) Nonparametric robustness (62G35) Estimation in multivariate analysis (62H12) Ridge regression; shrinkage estimators (Lasso) (62J07) Signal theory (characterization, reconstruction, filtering, etc.) (94A12)

Cites Work

• Nearly unbiased variable selection under minimax concave penalty
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• Title not available (Why is that?)
• One-step sparse estimates in nonconcave penalized likelihood models
• Simultaneous analysis of Lasso and Dantzig selector
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• High-dimensional graphs and variable selection with the Lasso
• Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies?
• Sure Independence Screening for Ultrahigh Dimensional Feature Space
• Large Covariance Estimation by Thresholding Principal Orthogonal Complements
• A unified framework for high-dimensional analysis of \(M\)-estimators with decomposable regularizers
• Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eQTL mapping
• Covariance regularization by thresholding
• Sparsistency and rates of convergence in large covariance matrix estimation
• Exact matrix completion via convex optimization
• Minimax Rates of Estimation for High-Dimensional Linear Regression Over $\ell_q$-Balls
• Estimation of (near) low-rank matrices with noise and high-dimensional scaling
• Nuclear-norm penalization and optimal rates for noisy low-rank matrix completion
• Geometric median and robust estimation in Banach spaces
• Challenging the empirical mean and empirical variance: a deviation study
• Statistical consistency and asymptotic normality for high-dimensional robust \(M\)-estimators
• Loss minimization and parameter estimation with heavy tails
• Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
• Inference and uncertainty quantification for noisy matrix completion
• Tight Oracle Inequalities for Low-Rank Matrix Recovery From a Minimal Number of Noisy Random Measurements
• Restricted strong convexity and weighted matrix completion: optimal bounds with noise
• Estimation of high-dimensional low-rank matrices
• Reconstruction From Anisotropic Random Measurements
• Adaptive thresholding for sparse covariance matrix estimation
• Estimation of High Dimensional Mean Regression in the Absence of Symmetry and Light Tail Assumptions
• Compressed sensing
• Optimal rates of convergence for sparse covariance matrix estimation
• Large covariance estimation through elliptical factor models
• Sparse Representation of a Polytope and Recovery of Sparse Signals and Low-Rank Matrices
• The restricted isometry property and its implications for compressed sensing
• ROP: matrix recovery via rank-one projections
• Atomic decomposition by basis pursuit
• A simpler approach to matrix completion
• Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising
• Concentration inequalities and moment bounds for sample covariance operators
• Robust estimators in high-dimensions without the computational intractability
• Empirical risk minimization for heavy-tailed losses
• On the prediction loss of the Lasso in the partially labeled setting
• High-dimensional robust precision matrix estimation: cellwise corruption under \(\epsilon \)-contamination
• Sub-Gaussian estimators of the mean of a random matrix with heavy-tailed entries
• Adaptive Huber Regression
• Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization

Cited In (25)

• D4R: doubly robust reduced rank regression in high dimension
• Low-rank matrix recovery under heavy-tailed errors
• High dimensional generalized linear models for temporal dependent data
• Adaptive robust large volatility matrix estimation based on high-frequency financial data
• Mining the factor zoo: estimation of latent factor models with sufficient proxies
• Robust signal dimension estimation via SURE
• Robust matrix estimations meet Frank-Wolfe algorithm
• The study of robust matrix regression models and algorithms
• ARFIS: an adaptive robust model for regression with heavy-tailed distribution
• Understanding Implicit Regularization in Over-Parameterized Single Index Model
• Robust high-dimensional tuning free multiple testing
• How do noise tails impact on deep ReLU networks?
• Robust covariance estimation for distributed principal component analysis
• Rate-optimal robust estimation of high-dimensional vector autoregressive models
• Robust inference for high‐dimensional single index models
• A framework of regularized low-rank matrix models for regression and classification
• Truthful and privacy-preserving generalized linear models
• Differential network inference via the fused D-trace loss with cross variables
• Dynamic Matrix Recovery
• Covariance Estimation for Matrix-valued Data
• Robust Recommendation via Social Network Enhanced Matrix Completion
• Large volatility matrix analysis using global and national factor models
• High-dimensional robust approximated \(M\)-estimators for mean regression with asymmetric data
• Robust parameter estimation of regression models under weakened moment assumptions
• User-friendly covariance estimation for heavy-tailed distributions

Uses Software

• PDCO

This page was built for publication: A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q820791)