Proximal Methods for Sparse Optimal Scoring and Discriminant Analysis
From MaRDI portal
Abstract: Linear discriminant analysis (LDA) is a classical method for dimensionality reduction, where discriminant vectors are sought to project data to a lower dimensional space for optimal separability of classes. Several recent papers have outlined strategies for exploiting sparsity for using LDA with high-dimensional data. However, many lack scalable methods for solution of the underlying optimization problems. We propose three new numerical optimization schemes for solving the sparse optimal scoring formulation of LDA based on block coordinate descent, the proximal gradient method, and the alternating direction method of multipliers. We show that the per-iteration cost of these methods scales linearly in the dimension of the data provided restricted regularization terms are employed, and cubically in the dimension of the data in the worst case. Furthermore, we establish that if our block coordinate descent framework generates convergent subsequences of iterates, then these subsequences converge to the stationary points of the sparse optimal scoring problem. We demonstrate the effectiveness of our new methods with empirical results for classification of Gaussian data and data sets drawn from benchmarking repositories, including time-series and multispectral X-ray data, and provide Matlab and R implementations of our optimization schemes.
Recommendations
- Sparse sufficient dimension reduction using optimal scoring
- Sparse discriminant analysis based on estimation of posterior probabilities
- Projection algorithms for nonconvex minimization with application to sparse principal component analysis
- Proximal methods for hierarchical sparse coding
- Sparse semiparametric discriminant analysis
- A direct approach to sparse discriminant analysis in ultra-high dimensions
- Sparse optimal discriminant clustering
- Asymptotic Optimality of Sparse Linear Discriminant Analysis with Arbitrary Number of Classes
- A convex optimization approach to high-dimensional sparse quadratic discriminant analysis
Cites work
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 6159604 (Why is no real title available?)
- scientific article; zbMATH DE number 6438182 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A direct approach to sparse discriminant analysis in ultra-high dimensions
- A direct estimation approach to sparse linear discriminant analysis
- Adaptive restart for accelerated gradient schemes
- Alternating direction method of multipliers for penalized zero-variance discriminant analysis
- Analysis and design of optimization algorithms via integral quadratic constraints
- Class prediction by nearest shrunken centroids, with applications to DNA microarrays.
- Distributed optimization and statistical learning via the alternating direction method of multipliers
- Fast alternating linearization methods for minimizing the sum of two convex functions
- First-order methods in optimization
- Flexible Discriminant Analysis by Optimal Scoring
- Gradient methods for minimizing composite functions
- High-dimensional classification using features annealed independence rules
- Least angle regression. (With discussion)
- Linear coupling: an ultimate unification of gradient and mirror descent
- Multiclass sparse discriminant analysis
- On the \(O(1/n)\) convergence rate of the Douglas-Rachford alternating direction method
- On the global and linear convergence of the generalized alternating direction method of multipliers
- Penalized classification using Fisher's linear discriminant
- Penalized discriminant analysis
- Regularization and Variable Selection Via the Elastic Net
- Smooth minimization of non-smooth functions
- Sparse linear discriminant analysis by thresholding for high dimensional data
- Spatial variation. 2nd ed
- The elements of statistical learning. Data mining, inference, and prediction
Cited in
(4)
This page was built for publication: Proximal Methods for Sparse Optimal Scoring and Discriminant Analysis
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q97534)
