Continuous time analysis of momentum methods
From MaRDI portal
Publication:4998888
Recommendations
- Steepest descent with momentum for quadratic functions is a version of the conjugate gradient method
- Stochastic dynamics of learning with momentum in neural networks
- On the influence of momentum acceleration on online learning
- Optimization with momentum: dynamical, control-theoretic, and symplectic perspectives
- Generalized momentum-based methods: a Hamiltonian perspective
Cites work
- A Stochastic Approximation Method
- Adaptive subgradient methods for online learning and stochastic optimization
- Analysis and design of optimization algorithms via integral quadratic constraints
- Applications of centre manifold theory
- Asymptotic Global Behavior for Stochastic Approximation and Diffusions with Slowly Decreasing Noise Effects: Global Minimization via Monte Carlo
- Bridging the gap between constant step size stochastic gradient descent and Markov chains
- Convergence of numerical time-averaging and stationary measures via Poisson equations
- Deep learning
- scientific article; zbMATH DE number 4015993 (Why is no real title available?)
- scientific article; zbMATH DE number 3850830 (Why is no real title available?)
- scientific article; zbMATH DE number 1269194 (Why is no real title available?)
- Introductory lectures on convex optimization. A basic course.
- Multiscale Methods
- Normally hyperbolic invariant manifolds in dynamical systems
- Numerical integrators based on modified differential equations
- On the Scope of the Method of Modified Equations
- Reducing the Dimensionality of Data with Neural Networks
- Semigroups of stochastic gradient descent and online principal component analysis: properties and diffusion approximations
- Stochastic approximation methods for constrained and unconstrained systems
- Stochastic heavy ball
- Stochastic modified equations and dynamics of stochastic gradient algorithms. I: Mathematical foundations
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
Cited in
(8)- How does momentum benefit deep neural networks architecture design? A few case studies
- Sampling Kaczmarz-Motzkin method for linear feasibility problems: generalization and acceleration
- Convergence of the Momentum Method for Semialgebraic Functions with Locally Lipschitz Gradients
- Uniform-in-time propagation of chaos for kinetic mean field Langevin dynamics
- Continuous Time Analysis of Momentum Methods
- Global stability of first-order methods for coercive tame functions
- Super level sets and exponential decay: a synergistic approach to stable neural network training
- Nesterov acceleration for ensemble Kalman inversion and variants
This page was built for publication: Continuous time analysis of momentum methods
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4998888)
