Nonsmooth nonconvex stochastic heavy ball

DOI10.1007/S10957-024-02408-3MaRDI QIDQ6536841zbMATH OpenFDO

Authors Tam Le

Publication date 14 May 2024

Published in Journal of Optimization Theory and Applications (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1007/S10957-024-02408-3

zbMATH Keywords

deep learningheavy ballnonsmooth optimizationo-minimal structurestochastic gradient

Mathematics Subject Classification ID

Set-valued set functions and measures; integration of set-valued functions; measurable selections (28B20) Nonsmooth analysis (49J52) Analysis of algorithms and problem complexity (68Q25) Online algorithms; streaming algorithms (68W27)

Cites work

• An inertial Newton algorithm for deep learning
• Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning
• Convergence of a stochastic subgradient method with averaging for nonsmooth nonconvex constrained optimization
• Convergence of constant step stochastic gradient descent for non-smooth non-convex functions
• Geometric categories and o-minimal structures
• scientific article; zbMATH DE number 41285 (Why is no real title available?)
• scientific article; zbMATH DE number 46303 (Why is no real title available?)
• scientific article; zbMATH DE number 1405930 (Why is no real title available?)
• Long term dynamics of the subgradient method for Lipschitz path differentiable functions
• Some methods of speeding up the convergence of iteration methods
• Stability under integration of sums of products of real globally subanalytic functions and their logarithms
• Stochastic Approximations and Differential Inclusions
• Stochastic generalized gradient method for nonconvex nonsmooth stochastic optimization
• Stochastic generalized-differentiable functions in the problem of nonconvex nonsmooth stochastic optimization
• Stochastic heavy ball
• Stochastic subgradient method converges on tame functions
• Topological aspects of nonsmooth optimization.

Cited in

(4)

• Stochastic Bregman subgradient methods for nonsmooth nonconvex optimization problems
• Learning-rate-free momentum SGD with reshuffling converges in nonsmooth nonconvex optimization
• Heavy-ball differential equation achieves \(O(\varepsilon^{-7/4})\) convergence for nonconvex functions
• The gradient's limit of a definable family of functions admits a variational stratification

This page was built for publication: Nonsmooth nonconvex stochastic heavy ball

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6536841)