Alternating direction method of multipliers for penalized zero-variance discriminant analysis
From MaRDI portal
(Redirected from Publication:97537)
Abstract: We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuristic, called sparse zero-variance discriminant analysis (SZVD), for simultaneously performing linear discriminant analysis and feature selection on high dimensional data. This method combines classical zero-variance discriminant analysis, where discriminant vectors are identified in the null space of the sample within-class covariance matrix, with penalization applied to induce sparse structures in the resulting vectors. To approximately solve the resulting nonconvex problem, we develop a simple algorithm based on the alternating direction method of multipliers. Further, we show that this algorithm is applicable to a larger class of penalized generalized eigenvalue problems, including a particular relaxation of the sparse principal component analysis problem. Finally, we establish theoretical guarantees for convergence of our algorithm to stationary points of the original nonconvex problem, and empirically demonstrate the effectiveness of our heuristic for classifying simulated data and data drawn from applications in time-series classification.
Recommendations
- A DC programming approach for sparse linear discriminant analysis
- Multiclass sparse discriminant analysis
- Penalized classification using Fisher's linear discriminant
- A direct approach to sparse discriminant analysis in ultra-high dimensions
- The Dantzig discriminant analysis with high dimensional data
Cites work
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- scientific article; zbMATH DE number 961607 (Why is no real title available?)
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- A direct estimation approach to sparse linear discriminant analysis
- A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis
- Alternating direction method of multipliers for sparse principal component analysis
- Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
- Approximation bounds for sparse principal component analysis
- Asymptotics of sample eigenstructure for a large dimensional spiked covariance model
- Class prediction by nearest shrunken centroids, with applications to DNA microarrays.
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
- Compressive sampling
- Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems
- Convex approximations to sparse PCA via Lagrangian duality
- Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Eigenvalues of large sample covariance matrices of spiked population models
- Generalized power method for sparse principal component analysis
- Improving implementation of linear discriminant analysis for the high dimension/small sample size problem
- Model Selection and Estimation in Regression with Grouped Variables
- Modified linear discriminant analysis approaches for classification of high-dimensional microarray data
- On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators
- Optimal solutions for sparse principal component analysis
- Penalized classification using Fisher's linear discriminant
- Penalized discriminant analysis
- Regularization and Variable Selection Via the Elastic Net
- Regularized linear discriminant analysis and its application in microarrays
- Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
- Sparse PCA: convex relaxations, algorithms and applications
- Sparse linear discriminant analysis by thresholding for high dimensional data
- Structured sparsity through convex optimization
- Theory and applications of compressed sensing
- Truncated power method for sparse eigenvalue problems
Cited in
(14)- Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
- The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates
- Splitting augmented Lagrangian-type algorithms with partial quadratic approximation to solve sparse signal recovery problems
- A DC programming approach for sparse linear discriminant analysis
- Alternating direction method of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreground extraction
- Convergence and rate analysis of a proximal linearized ADMM for nonconvex nonsmooth optimization
- Perturbed proximal primal-dual algorithm for nonconvex nonsmooth optimization
- Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis
- Decomposition methods for computing directional stationary solutions of a class of nonsmooth nonconvex optimization problems
- A partial Bregman ADMM with a general relaxation factor for structured nonconvex and nonsmooth optimization
- A proximal alternating direction method of multiplier for linearly constrained nonconvex minimization
- accSDA
- Convergence of Peaceman-Rachford splitting method with Bregman distance for three-block nonconvex nonseparable optimization
- Proximal Methods for Sparse Optimal Scoring and Discriminant Analysis
This page was built for publication: Alternating direction method of multipliers for penalized zero-variance discriminant analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q97537)
