An augmented Lagrangian approach for sparse principal component analysis
From MaRDI portal
Publication:715084
DOI10.1007/S10107-011-0452-4zbMATH Open1263.90093arXiv0907.2079OpenAlexW2104417092MaRDI QIDQ715084FDOQ715084
Publication date: 15 October 2012
Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)
Abstract: Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus often difficult to interpret the PCs. To alleviate this drawback, various sparse PCA approaches were proposed in literature [15, 6, 17, 28, 8, 25, 18, 7, 16]. Despite success in achieving sparsity, some important properties enjoyed by the standard PCA are lost in these methods such as uncorrelation of PCs and orthogonality of loading vectors. Also, the total explained variance that they attempt to maximize can be too optimistic. In this paper we propose a new formulation for sparse PCA, aiming at finding sparse and nearly uncorrelated PCs with orthogonal loading vectors while explaining as much of the total variance as possible. We also develop a novel augmented Lagrangian method for solving a class of nonsmooth constrained optimization problems, which is well suited for our formulation of sparse PCA. We show that it converges to a feasible point, and moreover under some regularity assumptions, it converges to a stationary point. Additionally, we propose two nonmonotone gradient methods for solving the augmented Lagrangian subproblems, and establish their global and local convergence. Finally, we compare our sparse PCA approach with several existing methods on synthetic, random, and real data, respectively. The computational results demonstrate that the sparse PCs produced by our approach substantially outperform those by other methods in terms of total explained variance, correlation of PCs, and orthogonality of loading vectors.
Full work available at URL: https://arxiv.org/abs/0907.2079
Recommendations
- Alternating direction method of multipliers for sparse principal component analysis
- Generalized power method for sparse principal component analysis
- The sparse principal component analysis problem: optimality conditions and algorithms
- Sparse principal component analysis and iterative thresholding
- Sparse principal component analysis via fractional function regularity
Factor analysis and principal components; correspondence analysis (62H25) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Numerical mathematical programming methods (65K05) Measures of association (correlation, canonical correlation, etc.) (62H20) Nonlinear programming (90C30)
Cites Work
- A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- The elements of statistical learning. Data mining, inference, and prediction
- Generalized power method for sparse principal component analysis
- Title not available (Why is that?)
- Nonmonotone Spectral Projected Gradient Methods on Convex Sets
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- Two-Point Step Size Gradient Methods
- An augmented Lagrangian approach for sparse principal component analysis
- Convex Analysis
- Sparse principal component analysis via regularized low rank matrix approximation
- A coordinate gradient descent method for nonsmooth separable minimization
- Sparse Reconstruction by Separable Approximation
- Nonlinear optimization.
- Title not available (Why is that?)
- Stability Theory for Systems of Inequalities, Part II: Differentiable Nonlinear Systems
- Optimization and dynamical systems
- Title not available (Why is that?)
Cited In (44)
- Exactly Uncorrelated Sparse Principal Component Analysis
- From simple structure to sparse components: a review
- Projected gradient approach to the numerical solution of the SCoTLASS
- A Randomized Nonmonotone Block Proximal Gradient Method for a Class of Structured Nonlinear Programming
- An approximate augmented Lagrangian method for nonnegative low-rank matrix approximation
- Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
- Do semidefinite relaxations solve sparse PCA up to the information limit?
- Projection onto a polyhedron that exploits sparsity
- An alternating direction method for finding Dantzig selectors
- Gradient Flows for Probabilistic Frame Potentials in the Wasserstein Space
- Convergence of a Class of Nonmonotone Descent Methods for Kurdyka–Łojasiewicz Optimization Problems
- Global convergence of ADMM in nonconvex nonsmooth optimization
- Fused Multiple Graphical Lasso
- Penalty and Augmented Lagrangian Methods for Constrained DC Programming
- Efficient method for symmetric nonnegative matrix factorization with an approximate augmented Lagrangian scheme
- Doubly iteratively reweighted algorithm for constrained compressed sensing models
- Augmented Lagrangian method for tensor low-rank and sparsity models in multi-dimensional image recovery
- A Proximal Point Dual Newton Algorithm for Solving Group Graphical Lasso Problems
- An augmented Lagrangian proximal alternating method for sparse discrete optimization problems
- A Lagrange–Newton algorithm for tensor sparse principal component analysis
- A semismooth Newton based augmented Lagrangian method for nonsmooth optimization on matrix manifolds
- Finding Dantzig selectors with a proximity operator based fixed-point algorithm
- A feasible method for optimization with orthogonality constraints
- PCA Sparsified
- Generalized left-localized Cayley parametrization for optimization with orthogonality constraints
- Riemannian trust region methods for \(\mathrm{SC}^1\) minimization
- A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
- Nonmonotone Barzilai-Borwein gradient algorithm for \(\ell_1\)-regularized nonsmooth minimization in compressive sensing
- Sparse Recovery via Partial Regularization: Models, Theory, and Algorithms
- Robust least square semidefinite programming with applications
- L1-norm-based principal component analysis with adaptive regularization
- Nonmonotone gradient methods for vector optimization with a portfolio optimization application
- Nonconvex and nonsmooth optimization with generalized orthogonality constraints: an approximate augmented Lagrangian method
- Principal component analysis with weighted sparsity constraint
- Two adaptive scaled gradient projection methods for Stiefel manifold constrained optimization
- Alternating direction method of multipliers for sparse principal component analysis
- An Augmented Lagrangian Method for Non-Lipschitz Nonconvex Programming
- Optimal detection of sparse principal components in high dimension
- An augmented Lagrangian approach for sparse principal component analysis
- An Augmented Lagrangian Method for $\ell_{1}$-Regularized Optimization Problems with Orthogonality Constraints
- Robust sparse principal component analysis
- A Newton-CG Based Augmented Lagrangian Method for Finding a Second-Order Stationary Point of Nonconvex Equality Constrained Optimization with Complexity Guarantees
- Sparse principal component analysis via regularized low rank matrix approximation
- Improve robustness of sparse PCA by \(L_{1}\)-norm maximization
Uses Software
This page was built for publication: An augmented Lagrangian approach for sparse principal component analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q715084)
