Covariance regularization by thresholding

DOI10.1214/08-AOS600zbMath1196.62062arXiv0901.3079OpenAlexW3099609308MaRDI QIDQ1000302

Publication date: 6 February 2009

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/0901.3079

zbMATH Keywords

regularization thresholding sparsity covariance estimation large $p$ small $n$climate data high dimension low sample size

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Estimation in multivariate analysis (62H12) Applications of statistics to environmental and related topics (62P12) Bootstrap, jackknife and other resampling methods (62F40)

Related Items

Covariance estimation via fiducial inference, Cholesky-based model averaging for covariance matrix estimation, Estimating a Change Point in a Sequence of Very High-Dimensional Covariance Matrices, Irrational Exuberance: Correcting Bias in Probability Estimates, Learning Latent Factors From Diversified Projections and Its Applications to Over-Estimated and Weak Factors, Cross-Validated Loss-based Covariance Matrix Estimator Selection in High Dimensions, \textit{ScreeNOT}: exact MSE-optimal singular value thresholding in correlated noise, On high-dimensional Poisson models with measurement error: hypothesis testing for nonlinear nonconvex optimization, Learning Gaussian graphical models with latent confounders, Statistical quality control using image intelligence: A sparse learning approach, Inference for Nonparanormal Partial Correlation via Regularized Rank-Based Nodewise Regression, On variable ordination of Cholesky‐based estimation for a sparse covariance matrix, On variable ordination of modified Cholesky decomposition for estimating time‐varying covariance matrices, A new approach for ultrahigh-dimensional covariance matrix estimation, An Efficient Greedy Search Algorithm for High-Dimensional Linear Discriminant Analysis, High-Dimensional Alpha Test of the Linear Factor Pricing Models With Heavy-Tailed Distributions, Robust Shape Matrix Estimation for High-Dimensional Compositional Data with Application to Microbial Inter-Taxa Analysis, Estimation of banded time-varying precision matrix based on SCAD and group Lasso, Structural factor equation models for causal network construction via directed acyclic mixed graphs, Covariance structure estimation with Laplace approximation, Tyler's and Maronna's M-estimators: non-asymptotic concentration results, An efficient GPU-parallel coordinate descent algorithm for sparse precision matrix estimation via scaled Lasso, Simultaneous test for linear model via projection, Conditional Functional Graphical Models, News-implied linkages and local dependency in the equity market, Testing and signal identification for two-sample high-dimensional covariances via multi-level thresholding, Large volatility matrix analysis using global and national factor models, Block-diagonal precision matrix regularization for ultra-high dimensional data, Post-processed posteriors for sparse covariances, Robust inference for high‐dimensional single index models, Covariance Estimation for Matrix-valued Data, Skilled Mutual Fund Selection: False Discovery Control Under Dependence, Optimal covariance matrix estimation for high-dimensional noise in high-frequency data, Time-varying minimum variance portfolio, Realized regression with asynchronous and noisy high frequency and high dimensional data, Dynamic Principal Component Analysis in High Dimensions, An Algebraic Estimator for Large Spectral Density Matrices, An adjoint-free four-dimensional variational data assimilation method via a modified Cholesky decomposition and an iterative Woodbury matrix formula, Positive-definite thresholding estimators of covariance matrices with zeros, A unified precision matrix estimation framework via sparse column-wise inverse operator under weak sparsity, Bayesian analysis of spherically parameterized dynamic multivariate stochastic volatility models, A LINEAR-PROGRAMMING PORTFOLIO OPTIMIZER TO MEAN–VARIANCE OPTIMIZATION, Large factor model estimation by nuclear norm plus $\ell_1$ norm penalization, The Lasso with general Gaussian designs with applications to hypothesis testing, Local Whittle estimation of high-dimensional long-run variance and precision matrices, Detection of Multiple Structural Breaks in Large Covariance Matrices, Robust Covariance Matrix Estimation for High-Dimensional Compositional Data with Application to Sales Data Analysis, Covariance Model with General Linear Structure and Divergent Parameters, Standard errors for panel data models with unknown clusters, LEARNING GRADIENTS FROM NONIDENTICAL DATA, FAST CONVERGENCE RATES IN ESTIMATING LARGE VOLATILITY MATRICES USING HIGH-FREQUENCY FINANCIAL DATA, Large Covariance Estimation by Thresholding Principal Orthogonal Complements, Trimmed estimators for large dimensional sparse covariance matrices, Projected principal component analysis in factor models, Bayesian inference via projections, A unified framework for high-dimensional analysis of $M$-estimators with decomposable regularizers, Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation, Discussion of ``Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation, Weak signals in high‐dimensional regression: Detection, estimation and prediction, Double shrinkage estimators for large sparse covariance matrices, Tuning-parameter selection in regularized estimations of large covariance matrices, Some Statistical Problems with High Dimensional Financial data, Spatially Dependent Multiple Testing Under Model Misspecification, With Application to Detection of Anthropogenic Influence on Extreme Climate Events, Large Covariance Estimation for Compositional Data Via Composition-Adjusted Thresholding, Graph-Guided Banding of the Covariance Matrix, Testing for Trends in High-Dimensional Time Series, Functions preserving positive definiteness for sparse matrices, Sparse Minimum Discrepancy Approach to Sufficient Dimension Reduction with Simultaneous Variable Selection in Ultrahigh Dimension, Smooth monotone covariance for elliptical distributions and applications in finance, Correlation structure selection for longitudinal data with diverging cluster size, Discussion: Latent variable graphical model selection via convex optimization, Discussion: Latent variable graphical model selection via convex optimization, Discussion: Latent variable graphical model selection via convex optimization, Discussion: Latent variable graphical model selection via convex optimization, Discussion: Latent variable graphical model selection via convex optimization, Estimation of a sparse and spiked covariance matrix, Estimation of high-dimensional dynamic conditional precision matrices with an application to forecast combination, The impact of jumps and leverage in forecasting covolatility, Multiple Anchor Point Shrinkage for the Sample Covariance Matrix, Preserving positivity for rank-constrained matrices, An improved banded estimation for large covariance matrix, Adaptive estimation of the copula correlation matrix for semiparametric elliptical copulas, A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery, Two-step sparse boosting for high-dimensional longitudinal data with varying coefficients, Scaling it up: stochastic search structure learning in graphical models, Empirical likelihood test for the equality of several high-dimensional covariance matrices, Estimation of linear functional of large spectral density matrix and application to Whittle's approach, Model-based clustering of high-dimensional data: a review, Low-rank diffusion matrix estimation for high-dimensional time-changed Lévy processes, Estimating sparse precision matrix: optimal rates of convergence and adaptive estimation, A self-calibrated direct approach to precision matrix estimation and linear discriminant analysis in high dimensions, Ensemble sparse estimation of covariance structure for exploring genetic disease data, Generalized principal component analysis for moderately non-stationary vector time series, Optimal sparse eigenspace and low-rank density matrix estimation for quantum systems, Estimating large correlation matrices for international migration, A testing based approach to the discovery of differentially correlated variable sets, Regularization for high-dimensional covariance matrix, More powerful tests for sparse high-dimensional covariances matrices, Matrix positivity preservers in fixed dimension. I, Nonparametric eigenvalue-regularized precision or covariance matrix estimator, Limiting spectral distribution of large-dimensional sample covariance matrices generated by VARMA, Testing super-diagonal structure in high dimensional covariance matrices, Robust inference of risks of large portfolios, Implied basket correlation dynamics, Sharp minimax tests for large covariance matrices and adaptation, Thresholding least-squares inference in high-dimensional regression models, Group-wise semiparametric modeling: a SCSE approach, Linear shrinkage estimation of large covariance matrices using factor models, Sparse PCA-based on high-dimensional Itô processes with measurement errors, Asymptotic theory for large volatility matrix estimation based on high-frequency financial data, On the systematic and idiosyncratic volatility with large panel high-frequency data, A regularized profile likelihood approach to covariance matrix estimation, Recent developments in high dimensional covariance estimation and its related issues, a review, High dimensional covariance matrix estimation by penalizing the matrix-logarithm transformed likelihood, Comparison of linear shrinkage estimators of a large covariance matrix in normal and non-normal distributions, Sparse estimation of high-dimensional correlation matrices, Asymptotic performance of PCA for high-dimensional heteroscedastic data, Sample size determination for high dimensional parameter estimation with application to biomarker identification, Estimating large covariance matrix with network topology for high-dimensional biomedical data, Minimax bounds for sparse PCA with noisy high-dimensional data, Confidence regions for entries of a large precision matrix, Using principal component analysis to estimate a high dimensional factor model with high-frequency data, Optimal detection of sparse principal components in high dimension, Optimal sparse volatility matrix estimation for high-dimensional Itô processes with measurement errors, Minimax optimal estimation of general bandable covariance matrices, Adjusting for high-dimensional covariates in sparse precision matrix estimation by $\ell_1$-penalization, PCA consistency for the power spiked model in high-dimensional settings, Forecasting co-volatilities via factor models with asymmetry and long memory in realized covariance, High-dimensional covariance matrix estimation with missing observations, Estimation of functionals of sparse covariance matrices, High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: risk underestimation, Retaining positive definiteness in thresholded matrices, Efficient estimation of approximate factor models via penalized maximum likelihood, Critical exponents of graphs, Estimation of the inverse scatter matrix of an elliptically symmetric distribution, Inference for high-dimensional differential correlation matrices, Nonstationary covariance modeling for incomplete data: Monte Carlo EM approach, High dimensional posterior convergence rates for decomposable graphical models, A Bayesian information criterion for portfolio selection, Covariance estimation: the GLM and regularization perspectives, High-dimensional covariance matrix estimation in approximate factor models, UPS delivers optimal phase diagram in high-dimensional variable selection, Detection of correlations, Covariance matrix estimation for stationary time series, Posterior convergence rates for estimating large precision matrices using graphical models, Spectral clustering via sparse graph structure learning with application to proteomic signaling networks in cancer, Covariance-insured screening, The spectral norm of random inner-product kernel matrices, Consistency of large dimensional sample covariance matrix under weak dependence, Complete characterization of Hadamard powers preserving Loewner positivity, monotonicity, and convexity, Efficient Bayesian regularization for graphical model selection, On nonparametric feature filters in electromagnetic imaging, Covariance structure regularization via Frobenius-norm discrepancy, Penalized least squares estimation with weakly dependent data, Shrinkage and model selection with correlated variables via weighted fusion, Test for bandedness of high-dimensional covariance matrices and bandwidth estimation, Distributed parameter and state estimation in petroleum reservoirs, Shrinkage tuning parameter selection in precision matrices estimation, Estimation of (near) low-rank matrices with noise and high-dimensional scaling, Sparse linear discriminant analysis by thresholding for high dimensional data, Autoregressive frequency detection using regularized least squares, Principal component analysis for second-order stationary vector time series, Partial estimation of covariance matrices, Optimal rates of convergence for covariance matrix estimation, Regularized estimation in sparse high-dimensional time series models, Estimating sufficient reductions of the predictors in abundant high-dimensional regressions, Confidence intervals for high-dimensional inverse covariance estimation, Estimation of covariance matrix via the sparse Cholesky factor with lasso, Geometric median and robust estimation in Banach spaces, Latent variable graphical model selection via convex optimization, Operator norm consistent estimation of large-dimensional sparse covariance matrices, Non-asymptotic error controlled sparse high dimensional precision matrix estimation, Minimax rate-optimal estimation of high-dimensional covariance matrices with incomplete data, Two sample tests for high-dimensional covariance matrices, Asymptotic normality and optimalities in estimation of large Gaussian graphical models, Honest confidence regions and optimality in high-dimensional precision matrix estimation, Optimal rates of convergence for sparse covariance matrix estimation, A penalized empirical likelihood method in high dimensions, Consistency of restricted maximum likelihood estimators of principal components, Sparsistency and rates of convergence in large covariance matrix estimation, Stable portfolio selection strategy for mean-variance-CVaR model under high-dimensional scenarios, Projection-based high-dimensional sign test, New hard-thresholding rules based on data splitting in high-dimensional imbalanced classification, High-dimensional sufficient dimension reduction through principal projections, Non-asymptotic properties of spectral decomposition of large Gram-type matrices and applications, Quadratic shrinkage for large covariance matrices, A generative approach to modeling data with quantitative and qualitative responses, Joint non-parametric estimation of mean and auto-covariances for Gaussian processes, Fitting Laplacian regularized stratified Gaussian models, Robust covariance estimation for distributed principal component analysis, A factor-GARCH model for high dimensional volatilities, Inference on covariance-mean regression, A numerical method for solving linear systems in the preconditioned Crank-Nicolson algorithm, Robust sparse covariance estimation by thresholding Tyler's M-estimator, Estimation of a multiplicative correlation structure in the large dimensional case, Discussion: Latent variable graphical model selection via convex optimization, Rejoinder: Latent variable graphical model selection via convex optimization, Adaptive covariance matrix estimation through block thresholding, Consistent Bayesian sparsity selection for high-dimensional Gaussian DAG models with multiplicative and beta-mixture priors, Multivariate tests of independence and their application in correlation analysis between financial markets, Statistical and computational limits for sparse matrix detection, Group symmetry and covariance regularization, Certifiably optimal sparse inverse covariance estimation, A dimension reduction based approach for estimation and variable selection in partially linear single-index models with high-dimensional covariates, Sparse permutation invariant covariance estimation, Structural shrinkage of nonparametric spectral estimators for multivariate time series, High dimensional sparse covariance estimation via directed acyclic graphs, Adaptive estimation of covariance matrices via Cholesky decomposition, High-dimensional covariance estimation by minimizing $\ell _{1}$-penalized log-determinant divergence, High dimensional classification for spatially dependent data with application to neuroimaging, Optimal rates of convergence for estimating Toeplitz covariance matrices, Double-slicing assisted sufficient dimension reduction for high-dimensional censored data, Minimax estimation of large precision matrices with bandable Cholesky factor, Estimating covariance and precision matrices along subspaces, Bootstrapping factor models with cross sectional dependence, Ill-posed estimation in high-dimensional models with instrumental variables, Robust covariance estimation for approximate factor models, Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data, Determination of vector error correction models in high dimensions, A multiple testing approach to the regularisation of large sample correlation matrices, Point process convergence for the off-diagonal entries of sample covariance matrices, The critical exponent: a novel graph invariant, An exponential inequality for U-statistics under mixing conditions, An efficient numerical method for condition number constrained covariance matrix approximation, On principal graphical models with application to gene network, Graph informed sliced inverse regression, Accuracy of regularized D-rule for binary classification, Large volatility matrix estimation with factor-based diffusion model for high-frequency financial data, Optimal estimation of a large-dimensional covariance matrix under Stein's loss, Covariance estimation via sparse Kronecker structures, On consistency and sparsity for sliced inverse regression in high dimensions, Gaussian and bootstrap approximations for high-dimensional U-statistics and their applications, Modeling correlated marker effects in genome-wide prediction via Gaussian concentration graph models, High-dimensional robust precision matrix estimation: cellwise corruption under $\epsilon $-contamination, Test for high dimensional covariance matrices, Projection tests for high-dimensional spiked covariance matrices, A note on moment inequality for quadratic forms, High-dimensional mean estimation via $\ell_1$ penalized normal likelihood, Weighted covariance matrix estimation, Fast and adaptive sparse precision matrix estimation in high dimensions, An efficient ADMM algorithm for high dimensional precision matrix estimation via penalized quadratic loss, Bayesian structure learning in graphical models, NOVELIST estimator of large correlation and covariance matrices and their inverses, Forecasting mortality rate improvements with a high-dimensional VAR, Nonparametric estimation of large covariance matrices with conditional sparsity, Power-law partial correlation network models, Estimation and optimal structure selection of high-dimensional Toeplitz covariance matrix, Spatial localization for nonlinear dynamical stochastic models for excitable media, Estimation of autocovariance matrices for high dimensional linear processes, Robust high-dimensional factor models with applications to statistical machine learning, Factor-adjusted multiple testing of correlations, An efficient algorithm for sparse inverse covariance matrix estimation based on dual formulation, Subspace estimation from unbalanced and incomplete data matrices: ${\ell_{2,\infty}}$ statistical guarantees, Bootstrap based inference for sparse high-dimensional time series models, Spatial disease mapping using directed acyclic graph auto-regressive (DAGAR) models, User-friendly covariance estimation for heavy-tailed distributions, Robust covariance and scatter matrix estimation under Huber's contamination model, Weak convergence of the empirical spectral distribution of high-dimensional band sample covariance matrices, A large covariance matrix estimator under intermediate spikiness regimes, Regularized estimation of precision matrix for high-dimensional multivariate longitudinal data, Multivariate variable selection by means of null-beamforming, Ultrahigh dimensional precision matrix estimation via refitted cross validation, Exponent of cross-sectional dependence for residuals, Compressed covariance estimation with automated dimension learning, Positive-definite modification of a covariance matrix by minimizing the matrix $\ell_{\infty}$ norm with applications to portfolio optimization, Bayesian discriminant analysis using a high dimensional predictor, Estimating high-dimensional covariance and precision matrices under general missing dependence, The finite sample properties of sparse M-estimators with pseudo-observations, Identifying the number of factors using a white noise test, Non-asymptotic rate for high-dimensional covariance estimation with non-independent missing observations, Design-free estimation of integrated covariance matrices for high-frequency data, Consistency of the objective general index in high-dimensional settings, The beta-mixture shrinkage prior for sparse covariances with near-minimax posterior convergence rate, Nonparametric variable screening for multivariate additive models, Unified discrete-time factor stochastic volatility and continuous-time Itô models for combining inference based on low-frequency and high-frequency, Regularized factor portfolio for cross-sectional multifactor models, Large sample correlation matrices: a comparison theorem and its applications, High-dimensional correlation matrix estimation for general continuous data with Bagging technique, New challenges in covariance estimation: multiple structures and coarse quantization, On skewed Gaussian graphical models, Covariance estimation under one-bit quantization, SURE-tuned tapering estimation of large covariance matrices, Sparse graphical models via calibrated concave convex procedure with application to fMRI data, A Cholesky-based estimation for large-dimensional covariance matrices, A Permutation Test for Two-Sample Means and Signal Identification of High-dimensional Data, Bayesian Estimation of Gaussian Conditional Random Fields, A Minimum Discrepancy Approach With Fourier Transform in Sufficient Dimension Reduction, An overview of heavy-tail extensions of multivariate Gaussian distribution and their relations, ESTIMATION OF THE KRONECKER COVARIANCE MODEL BY QUADRATIC FORM, Minimax optimal estimation of high-dimensional sparse covariance matrices with missing data, Bayesian sparse covariance decomposition with a graphical structure, Vector diffusion maps and the connection Laplacian, Vast volatility matrix estimation for high-frequency financial data, Joint Estimation of the Two-Level Gaussian Graphical Models Across Multiple Classes, Covariance-Regularized Regression and Classification for high Dimensional Problems, Estimating Large Precision Matrices via Modified Cholesky Decomposition, High-dimensional change-point estimation: combining filtering with convex optimization, A permutation-based Bayesian approach for inverse covariance estimation, Rates of convergence in conditional covariance matrix with nonparametric entries estimation, Unnamed Item, Inferences in panel data with interactive effects using large covariance matrices, A Greedy Algorithm for Sparse Precision Matrix Approximation, Convergence of covariance and spectral density estimates for high-dimensional locally stationary processes, The Dispersion Bias, Statistical inference for high-dimension, low-sample-size data, Pre-averaging estimate of high dimensional integrated covariance matrix with noisy and asynchronous high-frequency data, Detection of units with pervasive effects in large panel data models, Augmented factor models with applications to validating market risk factors and forecasting bond risk premia, High dimensional minimum variance portfolio estimation under statistical factor models, Differentially private high dimensional sparse covariance matrix estimation, Unnamed Item, Large covariance estimation through elliptical factor models, High-dimensional test for alpha in linear factor pricing models with sparse alternatives, On the penalized maximum likelihood estimation of high-dimensional approximate factor model, Fixed support positive-definite modification of covariance matrix estimators via linear shrinkage, Matrix means and a novel high-dimensional shrinkage phenomenon, Sparse covariance matrix estimation in high-dimensional deconvolution, Group-wise shrinkage estimation in penalized model-based clustering, Testing and support recovery of correlation structures for matrix-valued observations with an application to stock market data, Unnamed Item, Unnamed Item, Nonlinear Factor‐Augmented Predictive Regression Models with Functional Coefficients, Unnamed Item, Regression on manifolds: estimation of the exterior derivative, A guided random walk through some high dimensional problems, Detecting the large entries of a sparse covariance matrix in sub-quadratic time, Unnamed Item, Unnamed Item, Model-Free Forward Screening Via Cumulative Divergence, Optimal Sparse Linear Prediction for Block-missing Multi-modality Data Without Imputation, Structural inference in sparse high-dimensional vector autoregressions, Covariance selection by thresholding the sample correlation matrix, High-dimensional influence measure, Covariance and precision matrix estimation for high-dimensional time series, Sparse PCA: optimal rates and adaptive estimation, Constrained Factor Models for High-Dimensional Matrix-Variate Time Series, Asymptotic theory for maximum deviations of sample covariance matrix estimates, TESTING CONSTANCY OF CONDITIONAL VARIANCE IN HIGH DIMENSION, Design-free estimation of variance matrices, Random matrix theory in statistics: a review, A panorama of positivity. II: Fixed dimension, The Five Trolls Under the Bridge: Principal Component Analysis With Asynchronous and Noisy High Frequency Data, Positive-Definite ℓ₁-Penalized Estimation of Large Covariance Matrices, Statistical challenges of high-dimensional data, Sparse Matrix Graphical Models, Regularized Parameter Estimation in High-Dimensional Gaussian Mixture Models, An $\ell_{\infty}$ Eigenvector Perturbation Bound and Its Application to Robust Covariance Estimation, Simultaneous Clustering and Estimation of Heterogeneous Graphical Models, Sparse Estimation of Conditional Graphical Models With Application to Gene Networks, Informative Estimation and Selection of Correlation Structure for Longitudinal Data, On Distance-Based Permutation Tests for Between-Group Comparisons, Schur polynomials and matrix positivity preservers, Unnamed Item, Unnamed Item, AVERAGING OF AN INCREASING NUMBER OF MOMENT CONDITION ESTIMATORS, Embracing the Blessing of Dimensionality in Factor Models, Posterior contraction in sparse Bayesian factor models for massive covariance matrices, Sparse and low-rank covariance matrix estimation, Two-sample spatial rank test using projection, Estimating the Number of Sources in Magnetoencephalography Using Spiked Population Eigenvalues, Multilevel ensemble Kalman filtering, Efficient estimation of conditional covariance matrices for dimension reduction, On an Additive Semigraphoid Model for Statistical Networks With Application to Pathway Analysis, Selection of the Regularization Parameter in Graphical Models Using Network Characteristics, Source localization with MEG data: A beamforming approach based on covariance thresholding, Low-Rank Approximation and Completion of Positive Tensors, Banded Regularization of Autocovariance Matrices in Application to Parameter Estimation and Forecasting of Time Series, A new semiparametric estimation approach for large dynamic covariance matrices with multiple conditioning variables, Spectral analysis of high-dimensional time series, A Nonparametric Graphical Model for Functional Data With Application to Brain Networks Based on fMRI, Unnamed Item, High-dimensional autocovariance matrices and optimal linear prediction, Sparsity concepts and estimation procedures for high‐dimensional vector autoregressive models, Optimal estimation and rank detection for sparse spiked covariance matrices, Substitution principle for CLT of linear spectral statistics of high-dimensional sample covariance matrices with applications to hypothesis testing, Detecting positive correlations in a multivariate sample, On the sample covariance matrix estimator of reduced effective rank population matrices, with applications to fPCA, THE SPECTRUM OF RANDOM INNER-PRODUCT KERNEL MATRICES, ESTIMATION OF TIME-VARYING COVARIANCE MATRICES FOR LARGE DATASETS, An empirical estimator for the sparsity of a large covariance matrix under multivariate normal assumptions, Graph-Based Regularization for Regression Problems with Alignment and Highly Correlated Designs

Uses Software

Cites Work