An augmented Lagrangian approach for sparse principal component analysis
From MaRDI portal
Measures of association (correlation, canonical correlation, etc.) (62H20) Factor analysis and principal components; correspondence analysis (62H25) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Numerical mathematical programming methods (65K05) Nonlinear programming (90C30)
Abstract: Principal component analysis (PCA) is a widely used technique for data analysis and dimension reduction with numerous applications in science and engineering. However, the standard PCA suffers from the fact that the principal components (PCs) are usually linear combinations of all the original variables, and it is thus often difficult to interpret the PCs. To alleviate this drawback, various sparse PCA approaches were proposed in literature [15, 6, 17, 28, 8, 25, 18, 7, 16]. Despite success in achieving sparsity, some important properties enjoyed by the standard PCA are lost in these methods such as uncorrelation of PCs and orthogonality of loading vectors. Also, the total explained variance that they attempt to maximize can be too optimistic. In this paper we propose a new formulation for sparse PCA, aiming at finding sparse and nearly uncorrelated PCs with orthogonal loading vectors while explaining as much of the total variance as possible. We also develop a novel augmented Lagrangian method for solving a class of nonsmooth constrained optimization problems, which is well suited for our formulation of sparse PCA. We show that it converges to a feasible point, and moreover under some regularity assumptions, it converges to a stationary point. Additionally, we propose two nonmonotone gradient methods for solving the augmented Lagrangian subproblems, and establish their global and local convergence. Finally, we compare our sparse PCA approach with several existing methods on synthetic, random, and real data, respectively. The computational results demonstrate that the sparse PCs produced by our approach substantially outperform those by other methods in terms of total explained variance, correlation of PCs, and orthogonality of loading vectors.
The authors present a new approach for sparse principal component analysis (PCA) which is a successful method for data analysis and dimension reduction. They relate their approach to a new augmented Lagrange method for solving a family of nonsmooth constrained optimization problems. Finally, a sophisticated number of numerical test results show the advantages of this approach.
Recommendations
- Alternating direction method of multipliers for sparse principal component analysis
- Generalized power method for sparse principal component analysis
- The sparse principal component analysis problem: optimality conditions and algorithms
- Sparse principal component analysis and iterative thresholding
- Sparse principal component analysis via fractional function regularity
Cites work
- A coordinate gradient descent method for nonsmooth separable minimization
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization
- An augmented Lagrangian approach for sparse principal component analysis
- Convex Analysis
- Generalized power method for sparse principal component analysis
- scientific article; zbMATH DE number 439380 (Why is no real title available?)
- scientific article; zbMATH DE number 3821467 (Why is no real title available?)
- Nonlinear optimization.
- Nonmonotone Spectral Projected Gradient Methods on Convex Sets
- Optimal solutions for sparse principal component analysis
- Optimization and dynamical systems
- Sparse principal component analysis via regularized low rank matrix approximation
- Sparse Reconstruction by Separable Approximation
- Stability Theory for Systems of Inequalities, Part II: Differentiable Nonlinear Systems
- The elements of statistical learning. Data mining, inference, and prediction
- Two-Point Step Size Gradient Methods
Cited in
(58)- A feasible method for optimization with orthogonality constraints
- Finding Dantzig selectors with a proximity operator based fixed-point algorithm
- Nonmonotone gradient methods for vector optimization with a portfolio optimization application
- Global convergence of ADMM in nonconvex nonsmooth optimization
- An alternating direction method for finding Dantzig selectors
- Two adaptive scaled gradient projection methods for Stiefel manifold constrained optimization
- An approximate augmented Lagrangian method for nonnegative low-rank matrix approximation
- Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
- From simple structure to sparse components: a review
- An augmented Lagrangian proximal alternating method for sparse discrete optimization problems
- Robust sparse principal component analysis
- Do semidefinite relaxations solve sparse PCA up to the information limit?
- Nonconvex and nonsmooth optimization with generalized orthogonality constraints: an approximate augmented Lagrangian method
- L1-norm-based principal component analysis with adaptive regularization
- A new family of constrained principal component analysis (CPCA)
- Projection onto a polyhedron that exploits sparsity
- Generalized power method for sparse principal component analysis
- An augmented Lagrangian method for non-Lipschitz nonconvex programming
- The sparse principal component analysis problem: optimality conditions and algorithms
- Projection algorithms for nonconvex minimization with application to sparse principal component analysis
- An Augmented Lagrangian Method for $\ell_{1}$-Regularized Optimization Problems with Orthogonality Constraints
- rs-sparse principal component analysis: a mixed integer nonlinear programming approach with VNS
- Principal component analysis with weighted sparsity constraint
- Optimal detection of sparse principal components in high dimension
- Robust least square semidefinite programming with applications
- Alternating direction method of multipliers for sparse principal component analysis
- A stochastic variance reduction method for PCA by an exact penalty approach
- A randomized nonmonotone block proximal gradient method for a class of structured nonlinear programming
- Nonmonotone Barzilai-Borwein gradient algorithm for \(\ell_1\)-regularized nonsmooth minimization in compressive sensing
- A proximal point dual Newton algorithm for solving group graphical Lasso problems
- Sparse Recovery via Partial Regularization: Models, Theory, and Algorithms
- Fused multiple graphical lasso
- Penalty and augmented Lagrangian methods for constrained DC programming
- Doubly iteratively reweighted algorithm for constrained compressed sensing models
- A semismooth Newton based augmented Lagrangian method for nonsmooth optimization on matrix manifolds
- A Newton-CG Based Augmented Lagrangian Method for Finding a Second-Order Stationary Point of Nonconvex Equality Constrained Optimization with Complexity Guarantees
- Convergence of a Class of Nonmonotone Descent Methods for Kurdyka–Łojasiewicz Optimization Problems
- Gradient Flows for Probabilistic Frame Potentials in the Wasserstein Space
- PCA Sparsified
- Generalized left-localized Cayley parametrization for optimization with orthogonality constraints
- Convex approximations to sparse PCA via Lagrangian duality
- Improve robustness of sparse PCA by \(L_{1}\)-norm maximization
- Exactly Uncorrelated Sparse Principal Component Analysis
- Efficient method for symmetric nonnegative matrix factorization with an approximate augmented Lagrangian scheme
- Augmented Lagrangian method for tensor low-rank and sparsity models in multi-dimensional image recovery
- A Lagrange–Newton algorithm for tensor sparse principal component analysis
- Riemannian trust region methods for \(\mathrm{SC}^1\) minimization
- A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimization
- A first-order method for nonconvex-strongly-concave constrained minimax optimization
- Efficient low rank matrix recovery with flexible group sparse regularization
- On the robust isolated calmness of a class of nonsmooth optimizations on Riemannian manifolds and its applications
- An SQP-type proximal gradient method for composite optimization problems with equality constraints
- A first-order augmented Lagrangian method for constrained minimax optimization
- An augmented Lagrangian approach for sparse principal component analysis
- A non-monotone adaptive scaled gradient projection method for orthogonality constrained problems
- An inexact variable metric proximal linearization method for composite optimization on manifolds
- Sparse principal component analysis via regularized low rank matrix approximation
- Projected gradient approach to the numerical solution of the SCoTLASS
This page was built for publication: An augmented Lagrangian approach for sparse principal component analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q715084)
