Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems
From MaRDI portal
Abstract: In this paper, we introduce a stochastic projected subgradient method for weakly convex (i.e., uniformly prox-regular) nonsmooth, nonconvex functions---a wide class of functions which includes the additive and convex composite classes. At a high-level, the method is an inexact proximal point iteration in which the strongly convex proximal subproblems are quickly solved with a specialized stochastic projected subgradient method. The primary contribution of this paper is a simple proof that the proposed algorithm converges at the same rate as the stochastic gradient method for smooth nonconvex problems. This result appears to be the first convergence rate analysis of a stochastic (or even deterministic) subgradient method for the class of weakly convex functions.
Recommendations
- Stochastic model-based minimization of weakly convex functions
- Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization
- Stochastic proximal difference-of-convex algorithm with SPIDER for a class of nonconvex nonsmooth regularized problems
- A Stochastic Subgradient Method for Nonsmooth Nonconvex Multilevel Composition Optimization
- Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization
Cites work
- A Gauss-Newton method for convex composite optimization
- A Linearization Method for Nonsmooth Stochastic Programming Problems
- A model algorithm for composite nondifferentiable optimization problems
- A proximal method for composite minimization
- A redistributed proximal bundle method for nonconvex optimization
- A Stochastic Approximation Method
- Block stochastic gradient iteration for convex and nonconvex optimization
- Computing proximal points of nonconvex functions
- Descent methods for composite nondifferentiable optimization problems
- Error bounds, quadratic growth, and linear convergence of proximal methods
- Exact Recovery in the Stochastic Block Model
- Filling the gap between lower-\(C^1\) and lower-\(C^2\) functions
- scientific article; zbMATH DE number 3986407 (Why is no real title available?)
- scientific article; zbMATH DE number 3689160 (Why is no real title available?)
- scientific article; zbMATH DE number 3790208 (Why is no real title available?)
- scientific article; zbMATH DE number 46303 (Why is no real title available?)
- Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization
- Monotone Operators and the Proximal Point Algorithm
- Nonsmooth optimization using Taylor-like models: error bounds, convergence, and termination criteria
- Phase retrieval from very few measurements
- Robust Stochastic Approximation Approach to Stochastic Programming
- Second order necessary and sufficient conditions for convex composite NDO
- Solution of nonconvex nonsmooth stochastic optimization problems
- Stochastic compositional gradient descent: algorithms for minimizing compositions of expected-value functions
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
- Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization
- Stochastic Methods for Composite and Weakly Convex Optimization Problems
- Stochastic model-based minimization of weakly convex functions
- The nonsmooth landscape of phase retrieval
- Variational Analysis
Cited in
(48)- Subgradient methods for sharp weakly convex functions
- Inexact stochastic subgradient projection method for stochastic equilibrium problems with nonmonotone bifunctions: application to expected risk minimization in machine learning
- A zeroth order method for stochastic weakly convex optimization
- Variable metric proximal stochastic variance reduced gradient methods for nonconvex nonsmooth optimization
- Stochastic variance-reduced prox-linear algorithms for nonconvex composite optimization
- On strongly quasiconvex functions: existence results and proximal point algorithms
- A hybrid stochastic optimization framework for composite nonconvex optimization
- Complexity of an inexact proximal-point penalty method for constrained smooth non-convex optimization
- An online conjugate gradient algorithm for large-scale data analysis in machine learning
- Proximal methods avoid active strict saddles of weakly convex functions
- A stochastic subgradient method for distributionally robust non-convex and non-smooth learning
- A regularization interpretation of the proximal point method for weakly convex functions
- Stochastic AUC optimization with general loss
- Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization
- Nonasymptotic convergence of stochastic proximal point methods for constrained convex optimization
- Stochastic model-based minimization of weakly convex functions
- Stochastic proximal difference-of-convex algorithm with SPIDER for a class of nonconvex nonsmooth regularized problems
- scientific article; zbMATH DE number 7370566 (Why is no real title available?)
- A Stochastic Proximal Alternating Minimization for Nonsmooth and Nonconvex Optimization
- Proximal gradient methods with adaptive subspace sampling
- Weakly-convex-concave min-max optimization: provable algorithms and applications in machine learning
- Graphical convergence of subgradients in nonconvex optimization and learning
- Inexact proximal stochastic second-order methods for nonconvex composite optimization
- Stochastic conditional gradient++: (Non)convex minimization and continuous submodular maximization
- Convergence guarantees for a class of non-convex and non-smooth optimization problems
- A Single Timescale Stochastic Approximation Method for Nested Stochastic Optimization
- Generalized momentum-based methods: a Hamiltonian perspective
- Stochastic proximal linear method for structured non-convex problems
- Distributed stochastic inertial-accelerated methods with delayed derivatives for nonconvex problems
- Smoothed Variable Sample-Size Accelerated Proximal Methods for Nonsmooth Stochastic Convex Programs
- A Unified Analysis of Descent Sequences in Weakly Convex Optimization, Including Convergence Rates for Bundle Methods
- The landscape of the proximal point method for nonconvex-nonconcave minimax optimization
- Proximal stochastic recursive momentum algorithm for nonsmooth nonconvex optimization problems
- A semismooth Newton stochastic proximal point algorithm with variance reduction
- Convergence properties of stochastic proximal subgradient method in solving a class of composite optimization problems with cardinality regularizer
- Stochastic subgradient algorithm for nonsmooth nonconvex optimization
- Stochastic subgradient descent escapes active strict saddles on weakly convex functions
- No dimension-free deterministic algorithm computes approximate stationarities of Lipschitzians
- High probability bounds on AdaGrad for constrained weakly convex optimization
- Stochastic optimization over proximally smooth sets
- Accelerated minimax algorithms flock together
- Derivative-free stochastic bilevel optimization for inverse problems
- Error bounds, PL condition, and quadratic growth for weakly convex functions, and linear convergences of proximal point methods
- Stochastic optimization under hidden convexity
- First-order methods for nonsmooth nonconvex functional constrained optimization with or without Slater points
- Differentially private non-convex optimization under the KL condition with optimal rates
- Zeroth-order gradient and quasi-Newton methods for nonsmooth nonconvex stochastic optimization
- Equivariant denoisers for plug-and-play image restoration
This page was built for publication: Proximally guided stochastic subgradient method for nonsmooth, nonconvex problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5231692)
