Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
From MaRDI portal
Publication:5743151
Abstract: This paper deals with the estimation of a high-dimensional covariance with a conditional sparsity structure and fast-diverging eigenvalues. By assuming sparse error covariance matrix in an approximate factor model, we allow for the presence of some cross-sectional correlation even after taking out common but unobservable factors. We introduce the Principal Orthogonal complEment Thresholding (POET) method to explore such an approximate factor structure with sparsity. The POET estimator includes the sample covariance matrix, the factor-based covariance matrix (Fan, Fan, and Lv, 2008), the thresholding estimator (Bickel and Levina, 2008) and the adaptive thresholding estimator (Cai and Liu, 2011) as specific examples. We provide mathematical insights when the factor analysis is approximately the same as the principal component analysis for high-dimensional data. The rates of convergence of the sparse residual covariance matrix and the conditional sparse covariance matrix are studied under various norms. It is shown that the impact of estimating the unknown factors vanishes as the dimensionality increases. The uniform rates of convergence for the unobserved factors and their factor loadings are derived. The asymptotic results are also verified by extensive simulation studies. Finally, a real data application on portfolio allocation is presented.
Recommendations
- Large covariance estimation through elliptical factor models
- High dimensional covariance matrix estimation using a factor model
- High-dimensional covariance matrix estimation in approximate factor models
- Nonparametric estimation of large covariance matrices with conditional sparsity
- Asymptotics of empirical eigenstructure for high dimensional spiked covariance
Cites work
- scientific article; zbMATH DE number 1911755 (Why is no real title available?)
- scientific article; zbMATH DE number 3396952 (Why is no real title available?)
- A Singular Value Thresholding Algorithm for Matrix Completion
- A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets
- A general framework for multiple testing dependence
- A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis
- A two-step estimator for large approximate dynamic factor models based on Kalman filtering
- Adaptive thresholding for sparse covariance matrix estimation
- Arbitrage, Factor Structure, and Mean-Variance Analysis on Large Asset Markets
- Are more data always better for factor analysis?
- Asymptotics of sample eigenstructure for a large dimensional spiked covariance model
- Correlated \(z\)-values and the accuracy of large-scale statistical estimates
- Correlation and Large-Scale Simultaneous Significance Testing
- Covariance regularization by thresholding
- Determining the Number of Factors in Approximate Factor Models
- Determining the Number of Factors in the General Dynamic Factor Model
- Dynamic factors in the presence of blocks
- Estimation and Inference in Large Heterogeneous Panels with a Multifactor Error Structure
- Estimation with quadratic loss.
- Forecasting Using Principal Components From a Large Number of Predictors
- GMM estimation of linear panel data models with time-varying individual effects
- Generalized thresholding of large covariance matrices
- High dimensional covariance matrix estimation using a factor model
- High-dimensional analysis of semidefinite relaxations for sparse principal components
- High-dimensional covariance matrix estimation in approximate factor models
- High-dimensional graphs and variable selection with the Lasso
- High-dimensional sparse factor modeling: applications in gene expression genomics
- High-dimensional volatility matrix estimation via wavelets and thresholding
- Improved penalization for determining the number of factors in approximate factor models
- Inferential Theory for Factor Models of Large Dimensions
- Measure Theory and Probability Theory
- Minimax bounds for sparse PCA with noisy high-dimensional data
- Noisy matrix decomposition via convex relaxation: optimal rates in high dimensions
- Nonparametric modeling of longitudinal covariance structure in functional mapping of quantitative trait loci
- On consistency and sparsity for principal components analysis in high dimensions
- On the distribution of the largest eigenvalue in principal components analysis
- Optimal rates of convergence for sparse covariance matrix estimation
- Optimal solutions for sparse principal component analysis
- PCA consistency in high dimension, low sample size context
- Posterior contraction in sparse Bayesian factor models for massive covariance matrices
- Regularization of Wavelet Approximations
- Robust principal component analysis?
- Sparse principal component analysis and iterative thresholding
- Sparse principal component analysis via regularized low rank matrix approximation
- Sparsistency and rates of convergence in large covariance matrix estimation
- The Rotation of Eigenvectors by a Perturbation. III
- The econometrics of mean‐variance efficiency tests: a survey
- The generalized dynamic factor model consistency and rates
- Vast portfolio selection with gross-exposure constraints
Cited in
(only showing first 100 items - show all)- On determination of the number of factors in an approximate factor model
- A shrinkage principle for heavy-tailed data: high-dimensional robust low-rank matrix recovery
- Factor-driven two-regime regression
- Estimation of high-dimensional integrated covariance matrix based on noisy high-frequency data with multiple observations
- Binary response models for heterogeneous panel data with interactive fixed effects
- Optimal discriminant analysis in high-dimensional latent factor models
- Post-processed posteriors for sparse covariances
- A test of sphericity for high-dimensional data and its application for detection of divergently spiked noise
- Factor Extraction in Dynamic Factor Models: Kalman Filter Versus Principal Components
- Factor models for matrix-valued high-dimensional time series
- Preprocessing noisy functional data: a multivariate perspective
- High-dimensional two-sample mean vectors test and support recovery with factor adjustment
- Rank-based tests of cross-sectional dependence in panel data models
- Two-sample spatial rank test using projection
- Limiting laws for divergent spiked eigenvalues and largest nonspiked eigenvalue of sample covariance matrices
- Efficient estimation of heterogeneous coefficients in panel data models with common shocks
- scientific article; zbMATH DE number 7376764 (Why is no real title available?)
- Matrix-variate data analysis by two-way factor model with replicated observations
- Estimation of time-varying covariance matrices for large datasets
- Factor GARCH-Itô models for high-frequency data with application to large volatility matrix prediction
- Are Latent Factor Regression and Sparse Regression Adequate?
- Regularized estimation in sparse high-dimensional multivariate regression, with application to a DNA methylation study
- Optimal shrinkage estimator for high-dimensional mean vector
- Block-diagonal precision matrix regularization for ultra-high dimensional data
- Change-point testing for parallel data sets with FDR control
- Integrative Factor Regression and Its Inference for Multimodal Data Analysis
- On factor models with random missing: EM estimation, inference, and cross validation
- Testing Simultaneous Diagonalizability
- Direct shrinkage estimation of large dimensional precision matrix
- Tangency portfolio weights for singular covariance matrix in small and large dimensions: estimation and test theory
- Large covariance estimation for compositional data via composition-adjusted thresholding
- Testing for structural changes in factor models via a nonparametric regression
- Ridge-type linear shrinkage estimation of the mean matrix of a high-dimensional normal distribution
- Adaptive robust large volatility matrix estimation based on high-frequency financial data
- Inferences in panel data with interactive effects using large covariance matrices
- Rank determination in tensor factor model
- Knowing factors or factor loadings, or neither? Evaluating estimators of large covariance matrices with noisy and asynchronous data
- Structured volatility matrix estimation for non-synchronized high-frequency financial data
- Error covariance matrix estimation using ridge estimator
- Low-rank diffusion matrix estimation for high-dimensional time-changed Lévy processes
- A One-Sided Refined Symmetrized Data Aggregation Approach to Robust Mutual Fund Selection
- Inferential theory for generalized dynamic factor models
- Mining the factor zoo: estimation of latent factor models with sufficient proxies
- Power enhancement for testing multi-factor asset pricing models via Fisher's method
- Robustifying Markowitz
- Realized regression with asynchronous and noisy high frequency and high dimensional data
- Time-varying minimum variance portfolio
- An Algebraic Estimator for Large Spectral Density Matrices
- Selective Inference for Hierarchical Clustering
- Large volatility matrix estimation with factor-based diffusion model for high-frequency financial data
- Efficient estimation of approximate factor models via penalized maximum likelihood
- Rank regularized estimation of approximate factor models
- Estimating large correlation matrices for international migration
- Estimating large covariance matrix with network topology for high-dimensional biomedical data
- Semiparametric model for covariance regression analysis
- Adaptive test for mean vectors of high-dimensional time series data with factor structure
- Adaptive estimation in structured factor models with applications to overlapping clustering
- Detecting groups in large vector autoregressions
- Recursive estimation in large panel data models: theory and practice
- A new robust covariance matrix estimation for high-dimensional microbiome data
- High-dimensional covariance matrix estimation
- Adaptive thresholding for large volatility matrix estimation based on high-frequency financial data
- Tests of equal accuracy for nested models with estimated factors
- Spiked sample covariance matrices with possibly multiple bulk components
- Regularization for high-dimensional covariance matrix
- Testing against constant factor loading matrix with large panel high-frequency data
- Multiple Anchor Point Shrinkage for the Sample Covariance Matrix
- Noisy matrix completion: understanding statistical guarantees for convex relaxation via nonconvex optimization
- Estimation of large dimensional factor models with an unknown number of breaks
- A self-reliant projected information criterion for the number of factors
- Estimating latent asset-pricing factors
- scientific article; zbMATH DE number 7415082 (Why is no real title available?)
- scientific article; zbMATH DE number 7370530 (Why is no real title available?)
- A semiparametric latent factor model for large scale temporal data with heteroscedasticity
- Adaptive estimation in multivariate response regression with hidden variables
- High dimensional minimum variance portfolio estimation under statistical factor models
- Forecasting Conditional Covariance Matrices in High-Dimensional Time Series: A General Dynamic Factor Approach
- Estimation of Sparsity-Induced Weak Factor Models
- Inference in Sparsity-Induced Weak Factor Models
- High-Dimensional Factor Regression for Heterogeneous Subpopulations
- Embracing the blessing of dimensionality in factor models
- Asymptotics of empirical eigenstructure for high dimensional spiked covariance
- Bootstrapping factor models with cross sectional dependence
- A rank test for the number of factors with high-frequency data
- On variable ordination of Cholesky‐based estimation for a sparse covariance matrix
- High-dimensional covariance matrix estimation in approximate factor models
- Penalized Regression for Multiple Types of Many Features With Missing Data
- High-dimensional Markowitz portfolio optimization problem: empirical comparison of covariance matrix estimators
- Bayesian factor-adjusted sparse regression
- Sparse covariance matrix estimation by DCA-based algorithms
- Inference in latent factor regression with clusterable features
- Non-asymptotic properties of spectral decomposition of large Gram-type matrices and applications
- The five trolls under the bridge: principal component analysis with asynchronous and noisy high frequency data
- CDPA: common and distinctive pattern analysis between high-dimensional datasets
- Activation discovery with FDR control: application to fMRI data
- High-dimensional volatility matrix estimation with cross-sectional dependent and heavy-tailed microstructural noise
- Statistical quality control using image intelligence: A sparse learning approach
- Homogeneity and Structure Identification in Semiparametric Factor Models
- Estimation of a multiplicative correlation structure in the large dimensional case
- The two-to-infinity norm and singular subspace geometry with applications to high-dimensional statistics
This page was built for publication: Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5743151)
