Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
From MaRDI portal
Publication:5743151
DOI10.1111/RSSB.12016zbMATH Open1411.62138arXiv1201.0175OpenAlexW2040373108WikidataQ42559363 ScholiaQ42559363MaRDI QIDQ5743151FDOQ5743151
Authors: Yuan Liao, Martina Mincheva, Jianqing Fan 
Publication date: 9 May 2019
Published in: Journal of the Royal Statistical Society Series B: Statistical Methodology (Search for Journal in Brave)
Abstract: This paper deals with the estimation of a high-dimensional covariance with a conditional sparsity structure and fast-diverging eigenvalues. By assuming sparse error covariance matrix in an approximate factor model, we allow for the presence of some cross-sectional correlation even after taking out common but unobservable factors. We introduce the Principal Orthogonal complEment Thresholding (POET) method to explore such an approximate factor structure with sparsity. The POET estimator includes the sample covariance matrix, the factor-based covariance matrix (Fan, Fan, and Lv, 2008), the thresholding estimator (Bickel and Levina, 2008) and the adaptive thresholding estimator (Cai and Liu, 2011) as specific examples. We provide mathematical insights when the factor analysis is approximately the same as the principal component analysis for high-dimensional data. The rates of convergence of the sparse residual covariance matrix and the conditional sparse covariance matrix are studied under various norms. It is shown that the impact of estimating the unknown factors vanishes as the dimensionality increases. The uniform rates of convergence for the unobserved factors and their factor loadings are derived. The asymptotic results are also verified by extensive simulation studies. Finally, a real data application on portfolio allocation is presented.
Full work available at URL: https://arxiv.org/abs/1201.0175
Recommendations
- Large covariance estimation through elliptical factor models
- High dimensional covariance matrix estimation using a factor model
- High-dimensional covariance matrix estimation in approximate factor models
- Nonparametric estimation of large covariance matrices with conditional sparsity
- Asymptotics of empirical eigenstructure for high dimensional spiked covariance
principal componentssparse matrixhigh dimensionalitythresholdingapproximate factor modelcross-sectional correlationunknown factorslow rank matrixdiverging eigenvalues
Factor analysis and principal components; correspondence analysis (62H25) Estimation in multivariate analysis (62H12)
Cites Work
- A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis
- The generalized dynamic factor model consistency and rates
- Are more data always better for factor analysis?
- Forecasting Using Principal Components From a Large Number of Predictors
- Title not available (Why is that?)
- Inferential Theory for Factor Models of Large Dimensions
- Determining the Number of Factors in Approximate Factor Models
- A two-step estimator for large approximate dynamic factor models based on Kalman filtering
- High-dimensional graphs and variable selection with the Lasso
- A general framework for multiple testing dependence
- High dimensional covariance matrix estimation using a factor model
- Estimation and Inference in Large Heterogeneous Panels with a Multifactor Error Structure
- Determining the Number of Factors in the General Dynamic Factor Model
- Measure Theory and Probability Theory
- The Rotation of Eigenvectors by a Perturbation. III
- Improved penalization for determining the number of factors in approximate factor models
- Covariance regularization by thresholding
- Sparsistency and rates of convergence in large covariance matrix estimation
- On the distribution of the largest eigenvalue in principal components analysis
- Robust principal component analysis?
- A Singular Value Thresholding Algorithm for Matrix Completion
- Optimal solutions for sparse principal component analysis
- Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets
- Vast portfolio selection with gross-exposure constraints
- On consistency and sparsity for principal components analysis in high dimensions
- Correlation and Large-Scale Simultaneous Significance Testing
- Asymptotics of sample eigenstructure for a large dimensional spiked covariance model
- A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets
- Dynamic factors in the presence of blocks
- PCA consistency in high dimension, low sample size context
- Posterior contraction in sparse Bayesian factor models for massive covariance matrices
- Sparse principal component analysis and iterative thresholding
- Regularization of Wavelet Approximations
- High-dimensional sparse factor modeling: applications in gene expression genomics
- Sparse principal component analysis via regularized low rank matrix approximation
- Adaptive thresholding for sparse covariance matrix estimation
- Estimation with quadratic loss.
- Correlated \(z\)-values and the accuracy of large-scale statistical estimates
- Generalized thresholding of large covariance matrices
- Title not available (Why is that?)
- Optimal rates of convergence for sparse covariance matrix estimation
- Noisy matrix decomposition via convex relaxation: optimal rates in high dimensions
- GMM estimation of linear panel data models with time-varying individual effects
- High-dimensional covariance matrix estimation in approximate factor models
- High-dimensional volatility matrix estimation via wavelets and thresholding
- Minimax bounds for sparse PCA with noisy high-dimensional data
- High-dimensional analysis of semidefinite relaxations for sparse principal components
- Nonparametric modeling of longitudinal covariance structure in functional mapping of quantitative trait loci
- The econometrics of mean‐variance efficiency tests: a survey
Cited In (only showing first 100 items - show all)
- A test of sphericity for high-dimensional data and its application for detection of divergently spiked noise
- Matrix-variate data analysis by two-way factor model with replicated observations
- Limiting laws for divergent spiked eigenvalues and largest nonspiked eigenvalue of sample covariance matrices
- Factor GARCH-Itô models for high-frequency data with application to large volatility matrix prediction
- Optimal shrinkage estimator for high-dimensional mean vector
- Adaptive robust large volatility matrix estimation based on high-frequency financial data
- Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data
- Structured volatility matrix estimation for non-synchronized high-frequency financial data
- Low-rank diffusion matrix estimation for high-dimensional time-changed Lévy processes
- Estimating large correlation matrices for international migration
- Estimating large covariance matrix with network topology for high-dimensional biomedical data
- Semiparametric model for covariance regression analysis
- Adaptive estimation in structured factor models with applications to overlapping clustering
- Spiked sample covariance matrices with possibly multiple bulk components
- A self-reliant projected information criterion for the number of factors
- Title not available (Why is that?)
- Estimating latent asset-pricing factors
- Embracing the blessing of dimensionality in factor models
- On variable ordination of Cholesky‐based estimation for a sparse covariance matrix
- Penalized Regression for Multiple Types of Many Features With Missing Data
- High-dimensional Markowitz portfolio optimization problem: empirical comparison of covariance matrix estimators
- Sparse covariance matrix estimation by DCA-based algorithms
- High dimensional efficiency with applications to change point tests
- Forecasting large covariance matrix with high-frequency data using factor approach for the correlation matrix
- A new semiparametric estimation approach for large dynamic covariance matrices with multiple conditioning variables
- Estimation and optimal structure selection of high-dimensional Toeplitz covariance matrix
- Robust high-dimensional factor models with applications to statistical machine learning
- Transformed contribution ratio test for the number of factors in static approximate factor models
- Using principal component analysis to estimate a high dimensional factor model with high-frequency data
- TGCnA: temporal gene coexpression network analysis using a low-rank plus sparse framework
- Large-dimensional factor modeling based on high-frequency observations
- Recent developments in high dimensional covariance estimation and its related issues, a review
- A generalized likelihood ratio test for normal mean when \(p\) is greater than \(n\)
- Nonparametric estimation of large covariance matrices with conditional sparsity
- Bridging factor and sparse models
- Quasi maximum likelihood analysis of high dimensional constrained factor models
- A nonparametric eigenvalue-regularized integrated covariance matrix estimator for asset return data
- Factor-adjusted multiple testing of correlations
- Estimating change-point latent factor models for high-dimensional time series
- Mahalanobis metric based clustering for fixed effects model
- Recent advances in shrinkage-based high-dimensional inference
- Forecasting multiple time series with one-sided dynamic principal components
- Consistent estimation of high-dimensional factor models when the factor number is over-estimated
- A Bayesian-motivated test for high-dimensional linear regression models with fixed design matrix
- Positive-definite modification of a covariance matrix by minimizing the matrix \(\ell_{\infty}\) norm with applications to portfolio optimization
- Principal envelope model
- D-CCA: A Decomposition-Based Canonical Correlation Analysis for High-Dimensional Datasets
- Statistical analysis of sparse approximate factor models
- Tuning-parameter selection in regularized estimations of large covariance matrices
- Feature extraction for functional time series: theory and application to NIR spectroscopy data
- Consistently recovering the signal from noisy functional data
- Pivotal variable detection of the covariance matrix and its application to high-dimensional factor models
- Heterogeneity adjustment with applications to graphical model inference
- Title not available (Why is that?)
- Doubly debiased Lasso: high-dimensional inference under hidden confounding
- Binary response models for heterogeneous panel data with interactive fixed effects
- Factor Extraction in Dynamic Factor Models: Kalman Filter Versus Principal Components
- A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery
- Factor-driven two-regime regression
- Factor models for matrix-valued high-dimensional time series
- High-dimensional two-sample mean vectors test and support recovery with factor adjustment
- Rank-based tests of cross-sectional dependence in panel data models
- Block-diagonal precision matrix regularization for ultra-high dimensional data
- Change-point testing for parallel data sets with FDR control
- Large covariance estimation for compositional data via composition-adjusted thresholding
- Testing for structural changes in factor models via a nonparametric regression
- On factor models with random missing: EM estimation, inference, and cross validation
- Direct shrinkage estimation of large dimensional precision matrix
- Rank determination in tensor factor model
- Error covariance matrix estimation using ridge estimator
- Rank regularized estimation of approximate factor models
- Large volatility matrix estimation with factor-based diffusion model for high-frequency financial data
- Efficient estimation of approximate factor models via penalized maximum likelihood
- Adaptive test for mean vectors of high-dimensional time series data with factor structure
- Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization
- Adaptive thresholding for large volatility matrix estimation based on high-frequency financial data
- Testing against constant factor loading matrix with large panel high-frequency data
- Tests of equal accuracy for nested models with estimated factors
- Regularization for high-dimensional covariance matrix
- Estimation of large dimensional factor models with an unknown number of breaks
- High dimensional minimum variance portfolio estimation under statistical factor models
- Asymptotics of empirical eigenstructure for high dimensional spiked covariance
- Bootstrapping factor models with cross sectional dependence
- A rank test for the number of factors with high-frequency data
- Bayesian factor-adjusted sparse regression
- High-dimensional covariance matrix estimation in approximate factor models
- Sampling distributions of optimal portfolio weights and characteristics in small and large dimensions
- The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics
- Robust inference of risks of large portfolios
- Estimation and inference of change points in high-dimensional factor models
- Exact and asymptotic tests on a factor model in low and large dimensions with applications
- Generalized dynamic factor models and volatilities: estimation and forecasting
- Sparse covariance matrix estimation in high-dimensional deconvolution
- Extracting conditionally heteroskedastic components using independent component analysis
- On the systematic and idiosyncratic volatility with large panel high-frequency data
- Semiparametric Ultra-High Dimensional Model Averaging of Nonlinear Dynamic Time Series
- Linear shrinkage estimation of large covariance matrices using factor models
- Factor-Adjusted Regularized Model Selection
- High dimensional mean-variance optimization through factor analysis
- Large covariance estimation through elliptical factor models
Uses Software
This page was built for publication: Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5743151)
