A general system of differential equations to model first-order adaptive algorithms
From MaRDI portal
Recommendations
- Convergence and dynamical behavior of the ADAM algorithm for nonconvex stochastic optimization
- A differential equation for modeling Nesterov's accelerated gradient method: theory and insights
- Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance
- A Systematic Approach to Lyapunov Analyses of Continuous-Time Models in Convex Optimization
- From differential equation solvers to accelerated first-order methods for convex optimization
Cites work
- A differential equation for modeling Nesterov's accelerated gradient method: theory and insights
- A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics.
- A variational perspective on accelerated methods in optimization
- Adaptive subgradient methods for online learning and stochastic optimization
- Analysis and design of optimization algorithms via integral quadratic constraints
- Asymptotics for a gradient system with memory term
- Compound Quadrature Rules for the Product of Two Functions
- Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
- Gradient descent only converges to minimizers: non-isolated critical points and invariant regions
- scientific article; zbMATH DE number 5521647 (Why is no real title available?)
- scientific article; zbMATH DE number 1862452 (Why is no real title available?)
- Introductory lectures on convex optimization. A basic course.
- Long time behaviour and stationary regime of memory gradient diffusions
- On the long time behavior of second order differential equations with asymptotically small dissipation
- On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces
- Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3
- Stochastic heavy ball
Cited in
(9)- Power-law dynamic arising from machine learning
- Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance
- Convergence and dynamical behavior of the ADAM algorithm for nonconvex stochastic optimization
- An inertial Newton algorithm for deep learning
- Friction-adaptive descent: a family of dynamics-based optimization methods
- A Systematic Approach to Lyapunov Analyses of Continuous-Time Models in Convex Optimization
- A general system of differential equations to model first order adaptive algorithms
- A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization
- Convergence analysis of the ADAM algorithm for linear inverse problems
This page was built for publication: A general system of differential equations to model first-order adaptive algorithms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4969223)
