Abstract: This paper considers regularizing a covariance matrix of variables estimated from observations, by hard thresholding. We show that the thresholded estimate is consistent in the operator norm as long as the true covariance matrix is sparse in a suitable sense, the variables are Gaussian or sub-Gaussian, and , and obtain explicit rates. The results are uniform over families of covariance matrices which satisfy a fairly natural notion of sparsity. We discuss an intuitive resampling scheme for threshold selection and prove a general cross-validation result that justifies this approach. We also compare thresholding to other covariance estimators in simulations and on an example from climate data.
Recommendations
Cites work
- scientific article; zbMATH DE number 4213315 (Why is no real title available?)
- scientific article; zbMATH DE number 51763 (Why is no real title available?)
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- A well-conditioned estimator for large-dimensional covariance matrices
- Adapting to unknown sparsity by controlling the false discovery rate
- Asymptotics of cross-validated risk estimation in estimator selection and performance assess\-ment
- Asymptotics of sample eigenstructure for a large dimensional spiked covariance model
- Covariance matrix selection and estimation via penalised normal likelihood
- DISTRIBUTION OF EIGENVALUES FOR SOME SETS OF RANDOM MATRICES
- Empirical Bayes estimation of the multivariate normal covariance matrix
- Empirical Bayes selection of wavelet thresholds
- Estimation of a covariance matrix under Stein's loss
- Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter vari\-ants
- First-Order Methods for Sparse Covariance Selection
- High dimensional covariance matrix estimation using a factor model
- Ideal spatial adaptation by wavelet shrinkage
- Model selection and estimation in the Gaussian graphical model
- Network exploration via the adaptive LASSO and SCAD penalties
- Nonparametric estimation of large covariance matrices of longitudinal data
- On the distribution of the largest eigenvalue in principal components analysis
- Operator norm consistent estimation of large-dimensional sparse covariance matrices
- Regularized estimation of large covariance matrices
- Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
- Some theory for generalized boosting algorithms
- Sparse estimation of large covariance matrices via a nested Lasso penalty
- Sparse inverse covariance estimation with the graphical lasso
- Sparse permutation invariant covariance estimation
- Sparsistency and rates of convergence in large covariance matrix estimation
- Spectrum estimation for large dimensional covariance matrices using random matrix theory
- The concentration of measure phenomenon
- Tracy-Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
- Weak convergence and empirical processes. With applications to statistics
Cited in
(only showing first 100 items - show all)- An Inverse Norm Sign Test of Location Parameter for High-Dimensional Data
- Discussion of ``Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
- Estimating a Change Point in a Sequence of Very High-Dimensional Covariance Matrices
- Prediction in Locally Stationary Time Series
- Adjusting for high-dimensional covariates in sparse precision matrix estimation by \(\ell_1\)-penalization
- Minimax optimal estimation of general bandable covariance matrices
- Minimax optimal estimation of high-dimensional sparse covariance matrices with missing data
- Comment: Ridge Regression and Regularization of Large Matrices
- An Efficient Greedy Search Algorithm for High-Dimensional Linear Discriminant Analysis
- A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery
- Empirical likelihood test for the equality of several high-dimensional covariance matrices
- Estimation of linear functional of large spectral density matrix and application to Whittle's approach
- On consistency and sparsity for sliced inverse regression in high dimensions
- Bayesian inference via projections
- Smooth monotone covariance for elliptical distributions and applications in finance
- Operator norm consistent estimation of large-dimensional sparse covariance matrices
- Testing for trends in high-dimensional time series
- Identifying the number of factors using a white noise test
- Model-based clustering of high-dimensional data: a review
- Post-processed posteriors for sparse covariances
- Spatially dependent multiple testing under model misspecification, with application to detection of anthropogenic influence on extreme climate events
- Bayesian estimation of cluster covariance matrices of unknown form
- A permutation-based Bayesian approach for inverse covariance estimation
- A new approach for ultrahigh dimensional precision matrix estimation
- Local Whittle estimation of high-dimensional long-run variance and precision matrices
- Informative estimation and selection of correlation structure for longitudinal data
- Covariance-insured screening
- Structural shrinkage of nonparametric spectral estimators for multivariate time series
- Spectral clustering via sparse graph structure learning with application to proteomic signaling networks in cancer
- Shrinkage tuning parameter selection in precision matrices estimation
- Covariance estimation: the GLM and regularization perspectives
- UPS delivers optimal phase diagram in high-dimensional variable selection
- High-dimensional correlation matrix estimation for general continuous data with Bagging technique
- New challenges in covariance estimation: multiple structures and coarse quantization
- Regression on manifolds: estimation of the exterior derivative
- Covariance estimation under one-bit quantization
- A self-calibrated direct approach to precision matrix estimation and linear discriminant analysis in high dimensions
- Ensemble sparse estimation of covariance structure for exploring genetic disease data
- Double-slicing assisted sufficient dimension reduction for high-dimensional censored data
- Two-sample spatial rank test using projection
- Cholesky-based model averaging for covariance matrix estimation
- Statistical inference for high-dimension, low-sample-size data
- Optimal rates of convergence for covariance matrix estimation
- Estimation of high-dimensional dynamic conditional precision matrices with an application to forecast combination
- An improved banded estimation for large covariance matrix
- Sparse estimation of conditional graphical models with application to gene networks
- Convergence of covariance and spectral density estimates for high-dimensional locally stationary processes
- Estimation of time-varying covariance matrices for large datasets
- Adaptive Functional Thresholding for Sparse Covariance Function Estimation in High Dimensions
- Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
- Source localization with MEG data: a beamforming approach based on covariance thresholding
- Group symmetry and covariance regularization
- High dimensional sparse covariance estimation via directed acyclic graphs
- Sparse permutation invariant covariance estimation
- Block-diagonal precision matrix regularization for ultra-high dimensional data
- Autoregressive frequency detection using regularized least squares
- Noise covariance estimation in multi-task high-dimensional linear models
- Estimating Cell-Type-Specific Gene Co-Expression Networks from Bulk Gene Expression Data with an Application to Alzheimer’s Disease
- Simultaneous Decorrelation of Matrix Time Series
- Sparse basis covariance matrix estimation for high dimensional compositional data via hard thresholding
- Scaling it up: stochastic search structure learning in graphical models
- Large covariance estimation for compositional data via composition-adjusted thresholding
- Bootstrap based inference for sparse high-dimensional time series models
- Sharp minimax tests for large covariance matrices and adaptation
- Averaging of an increasing number of moment condition estimators
- Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients
- Inferences in panel data with interactive effects using large covariance matrices
- Efficient estimation of conditional covariance matrices for dimension reduction
- Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data
- Robust covariance estimation for approximate factor models
- Sparse estimation of high-dimensional correlation matrices
- Weighted covariance matrix estimation
- scientific article; zbMATH DE number 7255095 (Why is no real title available?)
- Low-rank diffusion matrix estimation for high-dimensional time-changed Lévy processes
- Determination of vector error correction models in high dimensions
- Ill-posed estimation in high-dimensional models with instrumental variables
- Statistical and computational limits for sparse matrix detection
- Discussion: Latent variable graphical model selection via convex optimization
- On principal graphical models with application to gene network
- Covariance-regularized regression and classification for high dimensional problems
- High-dimensional covariance estimation by minimizing \(\ell _{1}\)-penalized log-determinant divergence
- Skilled Mutual Fund Selection: False Discovery Control Under Dependence
- Optimal covariance matrix estimation for high-dimensional noise in high-frequency data
- Realized regression with asynchronous and noisy high frequency and high dimensional data
- Time-varying minimum variance portfolio
- An Algebraic Estimator for Large Spectral Density Matrices
- Dynamic Principal Component Analysis in High Dimensions
- Estimation of a sparse and spiked covariance matrix
- Adaptive thresholding for sparse covariance matrix estimation
- Bayesian sparse covariance decomposition with a graphical structure
- Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications
- Large volatility matrix estimation with factor-based diffusion model for high-frequency financial data
- High-dimensional robust precision matrix estimation: cellwise corruption under \(\epsilon \)-contamination
- Efficient estimation of approximate factor models via penalized maximum likelihood
- An adjoint-free four-dimensional variational data assimilation method via a modified Cholesky decomposition and an iterative Woodbury matrix formula
- A testing based approach to the discovery of differentially correlated variable sets
- Estimating large correlation matrices for international migration
- Functions preserving positive definiteness for sparse matrices
- The finite sample properties of sparse M-estimators with pseudo-observations
- An exponential inequality for U-statistics under mixing conditions
This page was built for publication: Covariance regularization by thresholding
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1000302)
