Stochastic subgradient method converges on tame functions
From MaRDI portal
Publication:2291732
DOI10.1007/s10208-018-09409-5zbMath1433.65141arXiv1804.07795OpenAlexW2964210434WikidataQ128619589 ScholiaQ128619589MaRDI QIDQ2291732
Damek Davis, Dmitriy Drusvyatskiy, Jason D. Lee, Sham M. Kakade
Publication date: 31 January 2020
Published in: Foundations of Computational Mathematics (Search for Journal in Brave)
Full work available at URL: https://arxiv.org/abs/1804.07795
Stochastic ordinary differential equations (aspects of stochastic analysis) (60H10) Ordinary differential inclusions (34A60) Stability and convergence of numerical methods for ordinary differential equations (65L20)
Related Items (41)
Convergence of constant step stochastic gradient descent for non-smooth non-convex functions ⋮ A framework for differential calculus on persistence barcodes ⋮ Graphical Convergence of Subgradients in Nonconvex Optimization and Learning ⋮ A forward-backward dynamical approach for nonsmooth problems with block structure coupled by a smooth function ⋮ Conservative and semismooth derivatives are equivalent for semialgebraic maps ⋮ A gradient sampling algorithm for stratified maps with applications to topological data analysis ⋮ Global convergence of the gradient method for functions definable in o-minimal structures ⋮ Lyapunov stability of the subgradient method with constant step size ⋮ Stochastic proximal subgradient descent oscillates in the vicinity of its accumulation set ⋮ Subgradient Sampling for Nonsmooth Nonconvex Minimization ⋮ An Improved Unconstrained Approach for Bilevel Optimization ⋮ Momentum-based variance-reduced proximal stochastic gradient method for composite nonconvex stochastic optimization ⋮ Stochastic approximation with discontinuous dynamics, differential inclusions, and applications ⋮ Certifying the Absence of Spurious Local Minima at Infinity ⋮ Pathological Subgradient Dynamics ⋮ Conservative parametric optimality and the ridge method for tame min-max problems ⋮ First-order methods for convex optimization ⋮ A Decomposition Algorithm for Two-Stage Stochastic Programs with Nonconvex Recourse Functions ⋮ MultiComposite Nonconvex Optimization for Training Deep Neural Networks ⋮ Generalized gradients in dynamic optimization, optimal control, and machine learning problems ⋮ Discussion of: ``Nonparametric regression using deep neural networks with ReLU activation function ⋮ Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization ⋮ Convergence and Dynamical Behavior of the ADAM Algorithm for Nonconvex Stochastic Optimization ⋮ Stochastic Approximation for Optimization in Shape Spaces ⋮ Determination of convex functions via subgradients of minimal norm ⋮ Manifold Sampling for Optimizing Nonsmooth Nonconvex Compositions ⋮ Stochastic proximal gradient methods for nonconvex problems in Hilbert spaces ⋮ Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning ⋮ Incremental without replacement sampling in nonconvex optimization ⋮ Every Local Minimum Value Is the Global Minimum Value of Induced Model in Nonconvex Machine Learning ⋮ Stochastic generalized gradient methods for training nonconvex nonsmooth neural networks ⋮ A stochastic approximation method for approximating the efficient frontier of chance-constrained nonlinear programs ⋮ An Inertial Newton Algorithm for Deep Learning ⋮ A Stochastic Subgradient Method for Nonsmooth Nonconvex Multilevel Composition Optimization ⋮ Unnamed Item ⋮ Ghost Penalties in Nonconvex Constrained Optimization: Diminishing Stepsizes and Iteration Complexity ⋮ Perturbed iterate SGD for Lipschitz continuous loss functions ⋮ The Structure of Conservative Gradient Fields ⋮ Asymptotic Properties of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization ⋮ Examples of Pathological Dynamics of the Subgradient Method for Lipschitz Path-Differentiable Functions ⋮ Four algorithms to solve symmetric multi-type non-negative matrix tri-factorization problem
Uses Software
Cites Work
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Unnamed Item
- Asymptotic convergence of nonlinear contraction semigroups in Hilbert space
- Geometric categories and o-minimal structures
- A function not constant on a connected set of critical points
- Curves of Descent
- Evaluating an element of the Clarke generalized Jacobian of a composite piecewise differentiable function
- A vector forward mode of automatic differentiation for generalized derivative evaluation
- Clarke Subgradients of Stratifiable Functions
- Critical values of set-valued maps with stratifiable graphs. Extensions of Sard and Smale-Sard theorems
- Robust Stochastic Approximation Approach to Stochastic Programming
- An Invitation to Tame Optimization
- Generalized Gradients and Applications
- Variational Analysis
- Lipschitz functions with maximal Clarke subdifferentials are generic
- Stochastic Methods for Composite and Weakly Convex Optimization Problems
- Stochastic Model-Based Minimization of Weakly Convex Functions
- Variational Analysis of Regular Mappings
- Stochastic Approximations and Differential Inclusions
- Stochastic Approximations and Differential Inclusions, Part II: Applications
- Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming
- Model completeness results for expansions of the ordered field of real numbers by restricted Pfaffian functions and the exponential function
- A Stochastic Approximation Method
This page was built for publication: Stochastic subgradient method converges on tame functions
