Optimization with momentum: dynamical, control-theoretic, and symplectic perspectives

MaRDI QIDQ4998976zbMATH OpenFDO

Authors Michael Muehlebach, Michael Jordan

Publication date 9 July 2021

Full work available at URL https://arxiv.org/abs/2002.12493, https://jmlr.csail.mit.edu/papers/v22/20-207.html

zbMATH Keywords

nonconvex optimizationsymplectic integrationgradient-based optimizationconvergence rate analysisNesterov acceleration

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• Generalized momentum-based methods: a Hamiltonian perspective
• Conformal symplectic and relativistic optimization
• A Lyapunov analysis of accelerated methods in optimization
• The rate of convergence of optimization algorithms obtained via discretizations of heavy ball dynamical systems for convex optimization problems
• On dissipative symplectic integration with applications to gradient-based optimization

Cites work

• scientific article; zbMATH DE number 3850830 (Why is no real title available?)
• scientific article; zbMATH DE number 48893 (Why is no real title available?)
• scientific article; zbMATH DE number 53382 (Why is no real title available?)
• scientific article; zbMATH DE number 88931 (Why is no real title available?)
• scientific article; zbMATH DE number 193048 (Why is no real title available?)
• scientific article; zbMATH DE number 3539473 (Why is no real title available?)
• scientific article; zbMATH DE number 1745051 (Why is no real title available?)
• scientific article; zbMATH DE number 2171469 (Why is no real title available?)
• scientific article; zbMATH DE number 1418991 (Why is no real title available?)
• A differential equation for modeling Nesterov's accelerated gradient method: theory and insights
• A trust region algorithm with a worst-case iteration complexity of \(\mathcal{O}(\epsilon ^{-3/2})\) for nonconvex optimization
• A variational perspective on accelerated methods in optimization
• Analysis and design of optimization algorithms via integral quadratic constraints
• Asymptotic analysis of differential equations
• Cubic regularization of Newton method and its global performance
• Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
• Introductory lectures on convex optimization. A basic course.
• Morse Theory. (AM-51)
• Nonlinear systems. Analysis, stability, and control
• Some methods of speeding up the convergence of iteration methods
• Stochastic heavy ball
• The approximate duality gap technique: a unified theory of first-order methods

Cited in

(8)

• DYNAMICAL, SYMPLECTIC AND STOCHASTIC PERSPECTIVES ON GRADIENT-BASED OPTIMIZATION
• Fast optimization via inertial dynamics with closed-loop damping
• scientific article; zbMATH DE number 7370630 (Why is no real title available?)
• Self-consistent dynamical field theory of kernel evolution in wide neural networks
• A control-theoretic perspective on optimal high-order optimization
• Decentralized concurrent learning with coordinated momentum and restart
• Optimization with Momentum: Dynamical, Control-Theoretic, and Symplectic Perspectives
• Accelerated optimization landscape of linear-quadratic regulator

This page was built for publication: Optimization with momentum: dynamical, control-theoretic, and symplectic perspectives

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4998976)