An augmented Lagrangian approach for sparse principal component analysis
From MaRDI portal
Factor analysis and principal components; correspondence analysis (62H25) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Numerical mathematical programming methods (65K05) Measures of association (correlation, canonical correlation, etc.) (62H20) Nonlinear programming (90C30)
Abstract: Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus often difficult to interpret the PCs. To alleviate this drawback, various sparse PCA approaches were proposed in literature [15, 6, 17, 28, 8, 25, 18, 7, 16]. Despite success in achieving sparsity, some important properties enjoyed by the standard PCA are lost in these methods such as uncorrelation of PCs and orthogonality of loading vectors. Also, the total explained variance that they attempt to maximize can be too optimistic. In this paper we propose a new formulation for sparse PCA, aiming at finding sparse and nearly uncorrelated PCs with orthogonal loading vectors while explaining as much of the total variance as possible. We also develop a novel augmented Lagrangian method for solving a class of nonsmooth constrained optimization problems, which is well suited for our formulation of sparse PCA. We show that it converges to a feasible point, and moreover under some regularity assumptions, it converges to a stationary point. Additionally, we propose two nonmonotone gradient methods for solving the augmented Lagrangian subproblems, and establish their global and local convergence. Finally, we compare our sparse PCA approach with several existing methods on synthetic, random, and real data, respectively. The computational results demonstrate that the sparse PCs produced by our approach substantially outperform those by other methods in terms of total explained variance, correlation of PCs, and orthogonality of loading vectors.
Recommendations
- Alternating direction method of multipliers for sparse principal component analysis
- Generalized power method for sparse principal component analysis
- The sparse principal component analysis problem: optimality conditions and algorithms
- Sparse principal component analysis and iterative thresholding
- Sparse principal component analysis via fractional function regularity
Cites work
- scientific article; zbMATH DE number 439380 (Why is no real title available?)
- scientific article; zbMATH DE number 3821467 (Why is no real title available?)
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A coordinate gradient descent method for nonsmooth separable minimization
- A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization
- An augmented Lagrangian approach for sparse principal component analysis
- Convex Analysis
- Generalized power method for sparse principal component analysis
- Nonlinear optimization.
- Nonmonotone Spectral Projected Gradient Methods on Convex Sets
- Optimal solutions for sparse principal component analysis
- Optimization and dynamical systems
- Sparse Reconstruction by Separable Approximation
- Sparse principal component analysis via regularized low rank matrix approximation
- Stability Theory for Systems of Inequalities, Part II: Differentiable Nonlinear Systems
- The elements of statistical learning. Data mining, inference, and prediction
- Two-Point Step Size Gradient Methods
Cited in
(51)- Robust least square semidefinite programming with applications
- Two adaptive scaled gradient projection methods for Stiefel manifold constrained optimization
- Projection algorithms for nonconvex minimization with application to sparse principal component analysis
- Robust sparse principal component analysis
- Exactly Uncorrelated Sparse Principal Component Analysis
- Alternating direction method of multipliers for sparse principal component analysis
- Finding Dantzig selectors with a proximity operator based fixed-point algorithm
- A Lagrange–Newton algorithm for tensor sparse principal component analysis
- Generalized power method for sparse principal component analysis
- L1-norm-based principal component analysis with adaptive regularization
- Do semidefinite relaxations solve sparse PCA up to the information limit?
- An approximate augmented Lagrangian method for nonnegative low-rank matrix approximation
- From simple structure to sparse components: a review
- Convex approximations to sparse PCA via Lagrangian duality
- Gradient Flows for Probabilistic Frame Potentials in the Wasserstein Space
- Global convergence of ADMM in nonconvex nonsmooth optimization
- Augmented Lagrangian method for tensor low-rank and sparsity models in multi-dimensional image recovery
- Nonconvex and nonsmooth optimization with generalized orthogonality constraints: an approximate augmented Lagrangian method
- A Newton-CG Based Augmented Lagrangian Method for Finding a Second-Order Stationary Point of Nonconvex Equality Constrained Optimization with Complexity Guarantees
- Convergence of a Class of Nonmonotone Descent Methods for Kurdyka–Łojasiewicz Optimization Problems
- A feasible method for optimization with orthogonality constraints
- A randomized nonmonotone block proximal gradient method for a class of structured nonlinear programming
- Optimal detection of sparse principal components in high dimension
- Projected gradient approach to the numerical solution of the SCoTLASS
- Principal component analysis with weighted sparsity constraint
- A stochastic variance reduction method for PCA by an exact penalty approach
- A proximal point dual Newton algorithm for solving group graphical Lasso problems
- Projection onto a polyhedron that exploits sparsity
- Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
- A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
- An augmented Lagrangian approach for sparse principal component analysis
- Penalty and augmented Lagrangian methods for constrained DC programming
- PCA Sparsified
- The sparse principal component analysis problem: optimality conditions and algorithms
- Generalized left-localized Cayley parametrization for optimization with orthogonality constraints
- Fused multiple graphical lasso
- Efficient method for symmetric nonnegative matrix factorization with an approximate augmented Lagrangian scheme
- Nonmonotone gradient methods for vector optimization with a portfolio optimization application
- An Augmented Lagrangian Method for $\ell_{1}$-Regularized Optimization Problems with Orthogonality Constraints
- A semismooth Newton based augmented Lagrangian method for nonsmooth optimization on matrix manifolds
- An augmented Lagrangian method for non-Lipschitz nonconvex programming
- Sparse Recovery via Partial Regularization: Models, Theory, and Algorithms
- \(rs\)-sparse principal component analysis: a mixed integer nonlinear programming approach with VNS
- Riemannian trust region methods for \(\mathrm{SC}^1\) minimization
- Doubly iteratively reweighted algorithm for constrained compressed sensing models
- Improve robustness of sparse PCA by \(L_{1}\)-norm maximization
- A new family of constrained principal component analysis (CPCA)
- Sparse principal component analysis via regularized low rank matrix approximation
- Nonmonotone Barzilai-Borwein gradient algorithm for \(\ell_1\)-regularized nonsmooth minimization in compressive sensing
- An alternating direction method for finding Dantzig selectors
- An augmented Lagrangian proximal alternating method for sparse discrete optimization problems
This page was built for publication: An augmented Lagrangian approach for sparse principal component analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q715084)
