Sparse principal component analysis and iterative thresholding

DOI10.1214/13-AOS1097zbMATH Open1267.62074arXiv1112.2432MaRDI QIDQ355104FDOQ355104

Authors: Zongming Ma 

Publication date: 24 July 2013

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: Principal component analysis (PCA) is a classical dimension reduction method which projects data onto the principal subspace spanned by the leading eigenvectors of the covariance matrix. However, it behaves poorly when the number of features p is comparable to, or even much larger than, the sample size n. In this paper, we propose a new iterative thresholding approach for estimating principal subspaces in the setting where the leading eigenvectors are sparse. Under a spiked covariance model, we find that the new approach recovers the principal subspace and leading eigenvectors consistently, and even optimally, in a range of high-dimensional sparse settings. Simulated examples also demonstrate its competitive performance.

Full work available at URL: https://arxiv.org/abs/1112.2432

Recommendations

• On consistency and sparsity for principal components analysis in high dimensions
• Sparse PCA: optimal rates and adaptive estimation
• Principal component analysis in very high-dimensional spaces
• Sparse PCA via covariance thresholding
• Minimax sparse principal subspace estimation in high dimensions

zbMATH Keywords

dimension reductionhigh-dimensional statisticssparsitythresholdingprincipal subspacespiked covariance model

Mathematics Subject Classification ID

Asymptotic properties of nonparametric inference (62G20) Factor analysis and principal components; correspondence analysis (62H25) Estimation in multivariate analysis (62H12)

Cites Work

• Analysis of Financial Time Series
• Functional data analysis.
• Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties
• A wavelet tour of signal processing. The sparse way.
• The Rotation of Eigenvectors by a Perturbation. III
• Title not available (Why is that?)
• On the distribution of the largest eigenvalue in principal components analysis
• Title not available (Why is that?)
• On consistency and sparsity for principal components analysis in high dimensions
• Asymptotics of the principal components estimator of large factor models with weakly influential factors
• Truncated power method for sparse eigenvalue problems
• Asymptotics of sample eigenstructure for a large dimensional spiked covariance model
• A Direct Formulation for Sparse PCA Using Semidefinite Programming
• Finite sample approximation results for principal component analysis: A matrix perturbation approach
• PCA consistency in high dimension, low sample size context
• Consistency of sparse PCA in high dimension, low sample size contexts
• Sparse principal component analysis and iterative thresholding
• Sparse principal component analysis via regularized low rank matrix approximation
• Perturbation bounds in connection with singular value decomposition
• Asymptotic Theory for Principal Component Analysis
• Unconditional bases are optimal bases for data compression and for statistical estimation
• A Gaussian scenario for unsupervised learning
• Sparse Variable PCA Using Geodesic Steepest Descent
• Learning Theory
• Discussion
• High-dimensional analysis of semidefinite relaxations for sparse principal components

Cited In (only showing first 100 items - show all)

• High-resolution signal recovery via generalized sampling and functional principal component analysis
• Sparse principal component analysis for high‐dimensional stationary time series
• Principal component analysis of binary data by iterated singular value decomposition
• Certifiably optimal sparse principal component analysis
• On consistency and sparsity for principal components analysis in high dimensions
• Title not available (Why is that?)
• Robust covariance estimation for approximate factor models
• Sparse power factorization: balancing peakiness and sample complexity
• Do semidefinite relaxations solve sparse PCA up to the information limit?
• Recovery of simultaneous low rank and two-way sparse coefficient matrices, a nonconvex approach
• Sparse generalized principal component analysis for large-scale applications beyond Gaussianity
• Universal latent space model fitting for large networks with edge covariates
• Hypothesis tests for large density matrices of quantum systems based on Pauli measurements
• Overlapping community detection in networks via sparse spectral decomposition
• Sparse PCA by iterative elimination algorithm
• Sparse principal component analysis by choice of norm
• A tight bound of hard thresholding
• Testing and estimating change-points in the covariance matrix of a high-dimensional time series
• Principal component analysis in very high-dimensional spaces
• Sparse principal component analysis with missing observations
• Optimal estimation and rank detection for sparse spiked covariance matrices
• Convergence rate of eigenvector empirical spectral distribution of large Wigner matrices
• Sparse PCA: optimal rates and adaptive estimation
• The spectral norm of random inner-product kernel matrices
• Sparse PCA via covariance thresholding
• Bernstein-von Mises theorems for functionals of the covariance matrix
• Random matrix theory in statistics: a review
• An \(\ell_{\infty}\) eigenvector perturbation bound and its application
• Estimating structured high-dimensional covariance and precision matrices: optimal rates and adaptive estimation
• Minimax estimation in sparse canonical correlation analysis
• Subspace estimation from unbalanced and incomplete data matrices: \({\ell_{2,\infty}}\) statistical guarantees
• Recent developments in high dimensional covariance estimation and its related issues, a review
• Minimax bounds for sparse PCA with noisy high-dimensional data
• Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem
• Statistical inference for principal components of spiked covariance matrices
• Estimation of functionals of sparse covariance matrices
• New asymptotic results in principal component analysis
• Optimal sparse eigenspace and low-rank density matrix estimation for quantum systems
• Convergence of eigenvector empirical spectral distribution of sample covariance matrices
• Large covariance estimation through elliptical factor models
• An empirical comparison of two approaches for CDPCA in high-dimensional data
• An Iterative Bayesian Algorithm for Sparse Component Analysis in Presence of Noise
• The benefit of group sparsity in group inference with de-biased scaled group Lasso
• ECA: High-Dimensional Elliptical Component Analysis in Non-Gaussian Distributions
• Scale-invariant sparse PCA on high-dimensional meta-elliptical data
• Projection sparse principal component analysis: an efficient least squares method
• Sparse PCA-based on high-dimensional Itô processes with measurement errors
• Simple Poisson PCA: an algorithm for (sparse) feature extraction with simultaneous dimension determination
• Large Covariance Estimation by Thresholding Principal Orthogonal Complements
• Sparse principal component analysis with missing observations
• Likelihood ratio test in multivariate linear regression: from low to high dimension
• Integrative sparse principal component analysis
• Projected principal component analysis in factor models
• Rate-optimal posterior contraction for sparse PCA
• Sparse principal component analysis and iterative thresholding
• Sparse principal component analysis in Hilbert space
• Optimal detection of sparse principal components in high dimension
• An augmented Lagrangian approach for sparse principal component analysis
• Robust covariance and scatter matrix estimation under Huber's contamination model
• Estimation of low-rank matrices via approximate message passing
• Testing equivalence of clustering
• Convergence rates of eigenvector empirical spectral distribution of large dimensional sample covariance matrix
• Modified Mahalanobis-Taguchi system based on proper orthogonal decomposition for high-dimensional-small-sample-size data classification
• Sparsistency and agnostic inference in sparse PCA
• Optimality and sub-optimality of PCA. I: Spiked random matrix models
• Near-optimal stochastic approximation for online principal component estimation
• Large dimensional analysis of general margin based classification methods
• Minimax sparse principal subspace estimation in high dimensions
• Eigenvectors from Eigenvalues Sparse Principal Component Analysis
• Automatic sparse principal component analysis
• Integrative Factor Regression and Its Inference for Multimodal Data Analysis
• Smoothing composite proximal gradient algorithm for sparse group Lasso problems with nonsmooth loss functions
• An efficient algorithm for Fantope-constrained sparse principal subspace estimation problem
• Dynamic Principal Component Analysis in High Dimensions
• Fast deflation sparse principal component analysis via subspace projections
• Smoothing algorithms for nonsmooth optimization over the Stiefel manifold with applications to the graph Fourier basis problem
• Provable sample-efficient sparse phase retrieval initialized by truncated power method
• Sparse Principal Component Analysis Based on Least Trimmed Squares
• Long random matrices and tensor unfolding
• Envelopes and principal component regression
• Sequential Scaled Sparse Factor Regression
• Fundamental limits of low-rank matrix estimation with diverging aspect ratios
• Title not available (Why is that?)
• A fast, provably accurate approximation algorithm for sparse principal component analysis reveals human genetic variation across the world
• Sparse equisigned PCA: algorithms and performance bounds in the noisy rank-1 setting
• Recovering PCA and sparse PCA via hybrid-\((\ell_1,\ell_2)\) sparse sampling of data elements
• Sparse principal component analysis based on transformed \({L_1}\) penalty
• Euclidean Representation of Low-Rank Matrices and Its Geometric Properties
• Linear spectral statistics of eigenvectors of anisotropic sample covariance matrices
• Compressed covariance estimation with automated dimension learning
• Scalable interpretable multi-response regression via SEED
• Partially Observed Dynamic Tensor Response Regression
• Trace Ratio Optimization for High-Dimensional Multi-Class Discrimination
• Large volatility matrix analysis using global and national factor models
• On Cross-Validation for Sparse Reduced Rank Regression
• A data-driven approach to conditional screening of high-dimensional variables
• Tensor canonical correlation analysis
• Large scale analysis of generalization error in learning using margin based classification methods
• Sparse constrained projection approximation subspace tracking
• A penalty-free infeasible approach for a class of nonsmooth optimization problems over the Stiefel manifold

Uses Software

• fda (R)
• FinTS
• PMA
• chemometrics

This page was built for publication: Sparse principal component analysis and iterative thresholding

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q355104)