Embracing the blessing of dimensionality in factor models
From MaRDI portal
Abstract: Factor modeling is an essential tool for exploring intrinsic dependence structures among high-dimensional random variables. Much progress has been made for estimating the covariance matrix from a high-dimensional factor model. However, the blessing of dimensionality has not yet been fully embraced in the literature: much of the available data is often ignored in constructing covariance matrix estimates. If our goal is to accurately estimate a covariance matrix of a set of targeted variables, shall we employ additional data, which are beyond the variables of interest, in the estimation? In this paper, we provide sufficient conditions for an affirmative answer, and further quantify its gain in terms of Fisher information and convergence rate. In fact, even an oracle-like result (as if all the factors were known) can be achieved when a sufficiently large number of variables is used. The idea of utilizing data as much as possible brings computational challenges. A divide-and-conquer algorithm is thus proposed to alleviate the computational burden, and also shown not to sacrifice any statistical accuracy in comparison with a pooled analysis. Simulation studies further confirm our advocacy for the use of full data, and demonstrate the effectiveness of the above algorithm. Our proposal is applied to a microarray data example that shows empirical benefits of using more data.
Recommendations
- High dimensional covariance matrix estimation using a factor model
- Factor modeling for high-dimensional time series: inference for the number of factors
- High dimensional covariance matrix estimation using multi-factor models from incomplete information
- Exploratory factor analysis -- parameter estimation and scores prediction with high-dimensional data
- Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
Cites work
- A Direct Approach to False Discovery Rates
- A Testing Procedure for Determining the Number of Factors in Approximate Factor Models With Large Datasets
- Adaptive thresholding for sparse covariance matrix estimation
- Common risk factors in the returns on stocks and bonds
- Covariance regularization by thresholding
- Determining the Number of Factors in Approximate Factor Models
- Efficient estimation of factor models
- Eigenvalue ratio test for the number of factors
- Factor modeling for high-dimensional time series: inference for the number of factors
- Generalized thresholding of large covariance matrices
- High dimensional covariance matrix estimation using a factor model
- High-dimensional covariance matrix estimation in approximate factor models
- High-dimensional sparse factor modeling: applications in gene expression genomics
- scientific article; zbMATH DE number 53182 (Why is no real title available?)
- Large covariance estimation by thresholding principal orthogonal complements. With discussion and authors' reply
- Nearly unbiased variable selection under minimax concave penalty
- Optimal rates of convergence for covariance matrix estimation
- Regularization of Wavelet Approximations
- Sparse permutation invariant covariance estimation
- Sparsistency and rates of convergence in large covariance matrix estimation
- Statistical analysis of factor models of high dimension
- Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
Cited in
(23)- A general framework for association analysis of heterogeneous data
- Robust high-dimensional factor models with applications to statistical machine learning
- Estimation and inference in semiparametric quantile factor models
- High-dimensional factor model and its applications to statistical machine learning
- Adjusting systematic bias in high dimensional principal component scores
- Advantages of Using Factorisation Machines as a Statistical Modelling Technique
- High-Dimensional Factor Regression for Heterogeneous Subpopulations
- Generalized Factor Model for Ultra-High Dimensional Correlated Variables with Mixed Types
- Integrative Factor Regression and Its Inference for Multimodal Data Analysis
- Sparse online principal component analysis for parameter estimation in factor model
- Are Latent Factor Regression and Sparse Regression Adequate?
- Homogeneity and Structure Identification in Semiparametric Factor Models
- Online change-point detection for matrix-valued time series with latent two-way factor structure
- High-dimensional overdispersed generalized factor model with application to single-cell sequencing data analysis
- Factor-adjusted tests for generalized linear models with multimodal data: an application to breast cancer data
- A refined bootstrap procedure for high-dimensional factor-augmented regression models
- Bayesian transfer learning
- Estimation and testing of expectile regression with efficient subsampling for massive data
- Generalized linear spectral statistics of high-dimensional sample covariance matrices and its applications
- Statistical Quantile Learning for Large Additive Latent Variable Models
- Data Thinning for Poisson Factor Models and its Applications
- High-dimensional inference for single-index models with latent factors
- Sparse generalized factor models with weaker loadings
This page was built for publication: Embracing the blessing of dimensionality in factor models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4690965)
