An augmented Lagrangian approach for sparse principal component analysis
From MaRDI portal
(Redirected from Publication:715084)
Factor analysis and principal components; correspondence analysis (62H25) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Numerical mathematical programming methods (65K05) Measures of association (correlation, canonical correlation, etc.) (62H20) Nonlinear programming (90C30)
Abstract: Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus often difficult to interpret the PCs. To alleviate this drawback, various sparse PCA approaches were proposed in literature [15, 6, 17, 28, 8, 25, 18, 7, 16]. Despite success in achieving sparsity, some important properties enjoyed by the standard PCA are lost in these methods such as uncorrelation of PCs and orthogonality of loading vectors. Also, the total explained variance that they attempt to maximize can be too optimistic. In this paper we propose a new formulation for sparse PCA, aiming at finding sparse and nearly uncorrelated PCs with orthogonal loading vectors while explaining as much of the total variance as possible. We also develop a novel augmented Lagrangian method for solving a class of nonsmooth constrained optimization problems, which is well suited for our formulation of sparse PCA. We show that it converges to a feasible point, and moreover under some regularity assumptions, it converges to a stationary point. Additionally, we propose two nonmonotone gradient methods for solving the augmented Lagrangian subproblems, and establish their global and local convergence. Finally, we compare our sparse PCA approach with several existing methods on synthetic, random, and real data, respectively. The computational results demonstrate that the sparse PCs produced by our approach substantially outperform those by other methods in terms of total explained variance, correlation of PCs, and orthogonality of loading vectors.
Recommendations
- Alternating direction method of multipliers for sparse principal component analysis
- Generalized power method for sparse principal component analysis
- The sparse principal component analysis problem: optimality conditions and algorithms
- Sparse principal component analysis and iterative thresholding
- Sparse principal component analysis via fractional function regularity
Cites work
- scientific article; zbMATH DE number 439380 (Why is no real title available?)
- scientific article; zbMATH DE number 3821467 (Why is no real title available?)
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A coordinate gradient descent method for nonsmooth separable minimization
- A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization
- An augmented Lagrangian approach for sparse principal component analysis
- Convex Analysis
- Generalized power method for sparse principal component analysis
- Nonlinear optimization.
- Nonmonotone Spectral Projected Gradient Methods on Convex Sets
- Optimal solutions for sparse principal component analysis
- Optimization and dynamical systems
- Sparse Reconstruction by Separable Approximation
- Sparse principal component analysis via regularized low rank matrix approximation
- Stability Theory for Systems of Inequalities, Part II: Differentiable Nonlinear Systems
- The elements of statistical learning. Data mining, inference, and prediction
- Two-Point Step Size Gradient Methods
Cited in
(51)- Projected gradient approach to the numerical solution of the SCoTLASS
- From simple structure to sparse components: a review
- Exactly Uncorrelated Sparse Principal Component Analysis
- An approximate augmented Lagrangian method for nonnegative low-rank matrix approximation
- Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
- Do semidefinite relaxations solve sparse PCA up to the information limit?
- Projection onto a polyhedron that exploits sparsity
- An alternating direction method for finding Dantzig selectors
- Gradient Flows for Probabilistic Frame Potentials in the Wasserstein Space
- Convergence of a Class of Nonmonotone Descent Methods for Kurdyka–Łojasiewicz Optimization Problems
- Global convergence of ADMM in nonconvex nonsmooth optimization
- Generalized power method for sparse principal component analysis
- Efficient method for symmetric nonnegative matrix factorization with an approximate augmented Lagrangian scheme
- Doubly iteratively reweighted algorithm for constrained compressed sensing models
- The sparse principal component analysis problem: optimality conditions and algorithms
- Augmented Lagrangian method for tensor low-rank and sparsity models in multi-dimensional image recovery
- An augmented Lagrangian proximal alternating method for sparse discrete optimization problems
- A Lagrange–Newton algorithm for tensor sparse principal component analysis
- A semismooth Newton based augmented Lagrangian method for nonsmooth optimization on matrix manifolds
- Finding Dantzig selectors with a proximity operator based fixed-point algorithm
- An augmented Lagrangian method for non-Lipschitz nonconvex programming
- A feasible method for optimization with orthogonality constraints
- rs-sparse principal component analysis: a mixed integer nonlinear programming approach with VNS
- A randomized nonmonotone block proximal gradient method for a class of structured nonlinear programming
- Penalty and augmented Lagrangian methods for constrained DC programming
- PCA Sparsified
- Generalized left-localized Cayley parametrization for optimization with orthogonality constraints
- Nonmonotone Barzilai-Borwein gradient algorithm for \(\ell_1\)-regularized nonsmooth minimization in compressive sensing
- Riemannian trust region methods for \(\mathrm{SC}^1\) minimization
- Fused multiple graphical lasso
- A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
- Robust least square semidefinite programming with applications
- L1-norm-based principal component analysis with adaptive regularization
- Sparse Recovery via Partial Regularization: Models, Theory, and Algorithms
- Nonmonotone gradient methods for vector optimization with a portfolio optimization application
- A new family of constrained principal component analysis (CPCA)
- Nonconvex and nonsmooth optimization with generalized orthogonality constraints: an approximate augmented Lagrangian method
- Projection algorithms for nonconvex minimization with application to sparse principal component analysis
- Principal component analysis with weighted sparsity constraint
- Alternating direction method of multipliers for sparse principal component analysis
- A proximal point dual Newton algorithm for solving group graphical Lasso problems
- Two adaptive scaled gradient projection methods for Stiefel manifold constrained optimization
- Convex approximations to sparse PCA via Lagrangian duality
- Optimal detection of sparse principal components in high dimension
- An augmented Lagrangian approach for sparse principal component analysis
- An Augmented Lagrangian Method for $\ell_{1}$-Regularized Optimization Problems with Orthogonality Constraints
- Robust sparse principal component analysis
- A stochastic variance reduction method for PCA by an exact penalty approach
- Sparse principal component analysis via regularized low rank matrix approximation
- A Newton-CG Based Augmented Lagrangian Method for Finding a Second-Order Stationary Point of Nonconvex Equality Constrained Optimization with Complexity Guarantees
- Improve robustness of sparse PCA by \(L_{1}\)-norm maximization
This page was built for publication: An augmented Lagrangian approach for sparse principal component analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q715084)
