Alternating direction method of multipliers for penalized zero-variance discriminant analysis
From MaRDI portal
Publication:97537
DOI10.48550/ARXIV.1401.5492zbMATH Open1352.90072arXiv1401.5492OpenAlexW2273602044MaRDI QIDQ97537FDOQ97537
Mingyi Hong, Brendan P. W. Ames, Brendan P. W. Ames, Mingyi Hong
Publication date: 21 January 2014
Published in: Computational Optimization and Applications (Search for Journal in Brave)
Abstract: We consider the task of classification in the high dimensional setting where the number of features of the given data is significantly greater than the number of observations. To accomplish this task, we propose a heuristic, called sparse zero-variance discriminant analysis (SZVD), for simultaneously performing linear discriminant analysis and feature selection on high dimensional data. This method combines classical zero-variance discriminant analysis, where discriminant vectors are identified in the null space of the sample within-class covariance matrix, with penalization applied to induce sparse structures in the resulting vectors. To approximately solve the resulting nonconvex problem, we develop a simple algorithm based on the alternating direction method of multipliers. Further, we show that this algorithm is applicable to a larger class of penalized generalized eigenvalue problems, including a particular relaxation of the sparse principal component analysis problem. Finally, we establish theoretical guarantees for convergence of our algorithm to stationary points of the original nonconvex problem, and empirically demonstrate the effectiveness of our heuristic for classifying simulated data and data drawn from applications in time-series classification.
Full work available at URL: https://arxiv.org/abs/1401.5492
feature selectionclassificationdimension reductionalternating direction method of multiplierslinear discriminant analysisnonconvex optimization
Cites Work
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Regularized linear discriminant analysis and its application in microarrays
- Penalized Classification using Fisher’s Linear Discriminant
- A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis
- Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers
- Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data
- Regularization and Variable Selection Via the Elastic Net
- Model Selection and Estimation in Regression with Grouped Variables
- Improving implementation of linear discriminant analysis for the high dimension/small sample size problem
- On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators
- Class prediction by nearest shrunken centroids, with applications to DNA microarrays.
- Some theory for Fisher's linear discriminant function, `naive Bayes', and some alternatives when there are many more variables than observations
- Penalized discriminant analysis
- Alternating maximization: unifying framework for 8 sparse PCA formulations and efficient parallel codes
- Eigenvalues of large sample covariance matrices of spiked population models
- Sparse PCA: Convex Relaxations, Algorithms and Applications
- Theory and applications of compressed sensing
- Generalized power method for sparse principal component analysis
- Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems
- A Direct Estimation Approach to Sparse Linear Discriminant Analysis
- Alternating direction method of multipliers for sparse principal component analysis
- Approximation bounds for sparse principal component analysis
- Discriminant Analysis with Singular Covariance Matrices: Methods and Applications to Spectroscopic Data
- Sparse linear discriminant analysis by thresholding for high dimensional data
- A Direct Formulation for Sparse PCA Using Semidefinite Programming
- Structured sparsity through convex optimization
- Convex approximations to sparse PCA via Lagrangian duality
- Modified linear discriminant analysis approaches for classification of high-dimensional microarray data
Cited In (13)
- Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
- Splitting augmented Lagrangian-type algorithms with partial quadratic approximation to solve sparse signal recovery problems
- Convergence and rate analysis of a proximal linearized ADMM for nonconvex nonsmooth optimization
- Perturbed proximal primal-dual algorithm for nonconvex nonsmooth optimization
- Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis
- A partial Bregman ADMM with a general relaxation factor for structured nonconvex and nonsmooth optimization
- Convergence of Peaceman-Rachford splitting method with Bregman distance for three-block nonconvex nonseparable optimization
- The Proximal Alternating Direction Method of Multipliers in the Nonconvex Setting: Convergence Analysis and Rates
- accSDA
- Proximal Methods for Sparse Optimal Scoring and Discriminant Analysis
- Decomposition Methods for Computing Directional Stationary Solutions of a Class of Nonsmooth Nonconvex Optimization Problems
- Alternating Direction Method of Multipliers for a Class of Nonconvex and Nonsmooth Problems with Applications to Background/Foreground Extraction
- A Proximal Alternating Direction Method of Multiplier for Linearly Constrained Nonconvex Minimization
Uses Software
This page was built for publication: Alternating direction method of multipliers for penalized zero-variance discriminant analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q97537)
