Accelerated First-Order Methods: Differential Equations and Lyapunov Functions
From MaRDI portal
Abstract: We develop a theory of accelerated first-order optimization from the viewpoint of differential equations and Lyapunov functions. Building upon the previous work of many researchers, we consider differential equations which model the behavior of accelerated gradient descent. Our main contributions are to provide a general framework for discretizating the differential equations to produce accelerated methods, and to provide physical intuition which helps explain the optimal damping rate. An important novelty is the generality of our approach, which leads to a unified derivation of a wide variety of methods, including versions of Nesterov's accelerated gradient descent, FISTA, and accelerated coordinate descent.
This page was built for publication: Accelerated First-Order Methods: Differential Equations and Lyapunov Functions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6315551)
