Alternating direction method of multipliers for penalized zero-variance discriminant analysis
From MaRDI portal
Publication:97537
DOI10.48550/ARXIV.1401.5492zbMATH Open1352.90072arXiv1401.5492OpenAlexW2273602044MaRDI QIDQ97537FDOQ97537
Authors: Brendan P. W. Ames, Mingyi Hong, Brendan P. W. Ames, Mingyi Hong 
Publication date: 21 January 2014
Published in: Computational Optimization and Applications (Search for Journal in Brave)
Abstract: We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuristic, called sparse zero-variance discriminant analysis (SZVD), for simultaneously performing linear discriminant analysis and feature selection on high dimensional data. This method combines classical zero-variance discriminant analysis, where discriminant vectors are identified in the null space of the sample within-class covariance matrix, with penalization applied to induce sparse structures in the resulting vectors. To approximately solve the resulting nonconvex problem, we develop a simple algorithm based on the alternating direction method of multipliers. Further, we show that this algorithm is applicable to a larger class of penalized generalized eigenvalue problems, including a particular relaxation of the sparse principal component analysis problem. Finally, we establish theoretical guarantees for convergence of our algorithm to stationary points of the original nonconvex problem, and empirically demonstrate the effectiveness of our heuristic for classifying simulated data and data drawn from applications in time-series classification.
Full work available at URL: https://arxiv.org/abs/1401.5492
Recommendations
- A DC programming approach for sparse linear discriminant analysis
- Multiclass sparse discriminant analysis
- Penalized classification using Fisher's linear discriminant
- A direct approach to sparse discriminant analysis in ultra-high dimensions
- The Dantzig discriminant analysis with high dimensional data
feature selectionclassificationdimension reductionalternating direction method of multiplierslinear discriminant analysisnonconvex optimization
Cites Work
- Regularized linear discriminant analysis and its application in microarrays
- Penalized classification using Fisher's linear discriminant
- A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis
- Title not available (Why is that?)
- Title not available (Why is that?)
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
- Regularization and Variable Selection Via the Elastic Net
- Model Selection and Estimation in Regression with Grouped Variables
- Title not available (Why is that?)
- Improving implementation of linear discriminant analysis for the high dimension/small sample size problem
- On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators
- Class prediction by nearest shrunken centroids, with applications to DNA microarrays.
- Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
- Penalized discriminant analysis
- Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
- Eigenvalues of large sample covariance matrices of spiked population models
- Sparse PCA: convex relaxations, algorithms and applications
- Theory and applications of compressed sensing
- Generalized power method for sparse principal component analysis
- Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems
- Optimal solutions for sparse principal component analysis
- A direct estimation approach to sparse linear discriminant analysis
- Alternating direction method of multipliers for sparse principal component analysis
- Approximation bounds for sparse principal component analysis
- Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data
- Truncated power method for sparse eigenvalue problems
- Asymptotics of sample eigenstructure for a large dimensional spiked covariance model
- Sparse linear discriminant analysis by thresholding for high dimensional data
- Compressive sampling
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- Structured sparsity through convex optimization
- Convex approximations to sparse PCA via Lagrangian duality
- Modified linear discriminant analysis approaches for classification of high-dimensional microarray data
Cited In (14)
- The proximal alternating direction method of multipliers in the nonconvex setting: convergence analysis and rates
- Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
- Splitting augmented Lagrangian-type algorithms with partial quadratic approximation to solve sparse signal recovery problems
- A DC programming approach for sparse linear discriminant analysis
- Alternating direction method of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreground extraction
- Convergence and rate analysis of a proximal linearized ADMM for nonconvex nonsmooth optimization
- Perturbed proximal primal-dual algorithm for nonconvex nonsmooth optimization
- Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis
- Decomposition methods for computing directional stationary solutions of a class of nonsmooth nonconvex optimization problems
- A partial Bregman ADMM with a general relaxation factor for structured nonconvex and nonsmooth optimization
- A proximal alternating direction method of multiplier for linearly constrained nonconvex minimization
- Convergence of Peaceman-Rachford splitting method with Bregman distance for three-block nonconvex nonseparable optimization
- accSDA
- Proximal Methods for Sparse Optimal Scoring and Discriminant Analysis
Uses Software
This page was built for publication: Alternating direction method of multipliers for penalized zero-variance discriminant analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q97537)
