Proximal gradient methods with adaptive subspace sampling
From MaRDI portal
Abstract: Many applications in machine learning or signal processing involve nonsmooth optimization problems. This nonsmoothness brings a low-dimensional structure to the optimal solutions. In this paper, we propose a randomized proximal gradient method harnessing this underlying structure. We introduce two key components: i) a random subspace proximal gradient algorithm; ii) an identification-based sampling of the subspaces. Their interplay brings a significant performance improvement on typical learning problems in terms of dimensions explored.
Recommendations
- Nonsmoothness in machine learning: specific structure, proximal identification, and applications
- Incremental proximal methods for large scale convex optimization
- Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems
- Adaptive subgradient methods for online learning and stochastic optimization
- Perturbed proximal primal-dual algorithm for nonconvex nonsmooth optimization
Cites work
- A Coordinate Descent Primal-Dual Algorithm and Application to Distributed Asynchronous Optimization
- A distributed flexible delay-tolerant proximal gradient algorithm
- A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints
- Accelerated block-coordinate relaxation for regularized optimization
- Active Sets, Nonsmoothness, and Sensitivity
- Activity identification and local linear convergence of forward-backward-type methods
- Convergence of a block coordinate descent method for nondifferentiable minimization
- Convex analysis and monotone operator theory in Hilbert spaces
- Convex optimization: algorithms and complexity
- Coordinate descent algorithms
- Coordinate descent with arbitrary sampling. I: Algorithms and complexity.
- De-noising by soft-thresholding
- Efficiency of coordinate descent methods on huge-scale optimization problems
- Enhancing sparsity by reweighted \(\ell _{1}\) minimization
- scientific article; zbMATH DE number 2155014 (Why is no real title available?)
- Identifiable Surfaces in Constrained Optimization
- Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function
- Model selection with low complexity priors
- On optimal probabilities in stochastic coordinate descent methods
- On the Goldstein-Levitin-Polyak gradient projection method
- On the Identification of Active Constraints
- Optimality, identifiability, and sensitivity
- Optimization with sparsity-inducing penalties
- Proximal splitting methods in signal processing
- Proximal Thresholding Algorithm for Minimization over Orthonormal Bases
- Sensitivity analysis for mirror-stratifiable convex functions
- Sparsity and Smoothness Via the Fused Lasso
Cited in
(10)- A stochastic subspace approach to gradient-free optimization in high dimensions
- Adaptive iterative Hessian sketch via A-optimal subsampling
- Subgradient and sampling algorithms for _1 regression
- Global optimization using random embeddings
- Random Coordinate Descent Methods for Nonseparable Composite Optimization
- A complexity analysis framework for active manifold identification with applications to L₀ and L_p regularization models
- A randomised non-descent method for global optimisation
- Quasi-Newton method with subspace gradients
- A second-order descent method with active-set prediction for group-sparse optimization
- Nonsmoothness in machine learning: specific structure, proximal identification, and applications
This page was built for publication: Proximal gradient methods with adaptive subspace sampling
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5026438)
