Asymptotic performance of PCA for high-dimensional heteroscedastic data

DOI10.1016/J.JMVA.2018.06.002zbMATH Open1395.62139DBLPjournals/ma/HongBF18arXiv1703.06610OpenAlexW2734385411WikidataQ91692254 ScholiaQ91692254MaRDI QIDQ1661372FDOQ1661372

David Hong, Laura Balzano, Jeffrey A. Fessler

Publication date: 16 August 2018

Published in: Journal of Multivariate Analysis (Search for Journal in Brave)

Abstract: Principal Component Analysis (PCA) is a classical method for reducing the dimensionality of data by projecting them onto a subspace that captures most of their variation. Effective use of PCA in modern applications requires understanding its performance for data that are both high-dimensional and heteroscedastic. This paper analyzes the statistical performance of PCA in this setting, i.e., for high-dimensional data drawn from a low-dimensional subspace and degraded by heteroscedastic noise. We provide simplified expressions for the asymptotic PCA recovery of the underlying subspace, subspace amplitudes and subspace coefficients; the expressions enable both easy and efficient calculation and reasoning about the performance of PCA. We exploit the structure of these expressions to show that, for a fixed average noise variance, the asymptotic recovery of PCA for heteroscedastic data is always worse than that for homoscedastic data (i.e., for noise variances that are equal across samples). Hence, while average noise variance is often a practically convenient measure for the overall quality of data, it gives an overly optimistic estimate of the performance of PCA for heteroscedastic data.

Full work available at URL: https://arxiv.org/abs/1703.06610

Recommendations

• Heteroskedastic PCA: algorithm, optimality, and applications
• Nonasymptotic upper bounds for the reconstruction error of PCA
• PCA consistency in high dimension, low sample size context
• On consistency and sparsity for principal components analysis in high dimensions
• Principal component analysis in very high-dimensional spaces

zbMATH Keywords

heteroscedasticityhigh-dimensional dataprincipal component analysissubspace estimationasymptotic random matrix theory

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Factor analysis and principal components; correspondence analysis (62H25) Estimation in multivariate analysis (62H12)

Cites Work

• The polynomial method for random matrices
• OptShrink: An Algorithm for Improved Low-Rank Signal Matrix Denoising by Optimal, Data-Driven Singular Value Shrinkage
• Principal component analysis.
• Robust Statistics
• Covariance regularization by thresholding
• Spectral analysis of large dimensional random matrices
• On the distribution of the largest eigenvalue in principal components analysis
• Matrix estimation by universal singular value thresholding
• Robust principal component analysis?
• Robust Estimation of Dispersion Matrices and Principal Components
• On Consistency and Sparsity for Principal Components Analysis in High Dimensions
• Title not available (Why is that?)
• High breakdown estimators for principal components: the projection-pursuit approach revis\-ited
• Finite sample approximation results for principal component analysis: A matrix perturbation approach
• The singular values and vectors of low rank perturbations of large rectangular random matrices
• Probabilistic Principal Component Analysis
• Operator norm consistent estimation of large-dimensional sparse covariance matrices
• Rank-Sparsity Incoherence for Matrix Decomposition
• On sample eigenvalues in a generalized spiked population model
• Robust PCA via Outlier Pursuit
• Statistical challenges of high-dimensional data
• Robust computation of linear models by convex relaxation
• Large Sample Covariance Matrices and High-Dimensional Data Analysis
• Strong convergence of the empirical distribution of eigenvalues of sample covariance matrices with a perturbation matrix
• Asymptotic performance of PCA for high-dimensional heteroscedastic data
• Recursive Robust PCA or Recursive Sparse Recovery in Large but Structured Noise
• Statistical mechanics of unsupervised structure recognition
• Asymptotic Conditional Singular Value Decomposition for High-Dimensional Genomic Data

Cited In (14)

• Factor Extraction in Dynamic Factor Models: Kalman Filter Versus Principal Components
• Rapid evaluation of the spectral signal detection threshold and Stieltjes transform
• Boundary behavior in high dimension, low sample size asymptotics of PCA
• On the non-asymptotic concentration of heteroskedastic Wishart-type matrix
• \textit{ScreeNOT}: exact MSE-optimal singular value thresholding in correlated noise
• Biwhitening Reveals the Rank of a Count Matrix
• Optimally Weighted PCA for High-Dimensional Heteroscedastic Data
• Variance variation criterion and consistency in estimating the number of significant signals of high-dimensional PCA
• Asymptotic performance of PCA for high-dimensional heteroscedastic data
• Stochastic Gradients for Large-Scale Tensor Decomposition
• Matrix Denoising for Weighted Loss Functions and Heterogeneous Signals
• Heteroskedastic PCA: algorithm, optimality, and applications
• A note on identifiability conditions in confirmatory factor analysis
• Asymptotic Distribution of Studentized Contribution Ratio in High-Dimensional Principal Component Analysis

Uses Software

• OptShrink

This page was built for publication: Asymptotic performance of PCA for high-dimensional heteroscedastic data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1661372)