Analysis of continuous dynamical system models with Hessians derived from optimization methods
The authors consider continuous-time dynamical systems involving Hessians, derived from unconstrained continuous optimization methods, \(x_{\ast}\in\arg\min_{x\in \mathbb{R}^{n}}f(x)\), of the form: \[ a_{1}\ddot{x}+a_{2}\dot{x}+a_{3}\nabla^{2}f\dot{x}+a_{4}\nabla f=0 \tag{1} \] where \(a_{i}:\mathbb{R}^{+}\rightarrow \mathbb{R}\) \((i=1,2,3,4)\) are some functions of \(t\), and \(a_{3}\) and \(a_{4}\) are not zeros. They classify \((1)\) into three cases, and for each case, they exhibit a Lyapunov function and obtain the best rate of convergence when using this Lyapunov function. However, in some cases, this rate is the same as either the simplest gradient flow, or the continuous dynamical system corresponding to Nesterov's accelerated gradient method. Therefore in these cases, the incorporation of the Hessian does not bring any improvement. Their results extend previous results in [\textit{H. Attouch} et al., J. Differ. Equations 261, No. 10, 5734--5783 (2016; Zbl 1375.49028); \textit{B. Shi} et al., Math. Program. 195, No. 1--2 (A), 79--148 (2022; Zbl 1500.65026); \textit{J. J. Suh} et al., ``Continuous-time analysis of accelerated gradient methods via conservation laws in dilated coordinate systems, Preprint, \url{arXiv:2202.05501}].
- The rate of convergence of optimization algorithms obtained via discretizations of heavy ball dynamical systems for convex optimization problems
- A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics.
- Approaching nonsmooth nonconvex optimization problems through first order dynamical systems with hidden acceleration and Hessian driven damping terms
- On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping
- Fast convergence of inertial dynamics with Hessian-driven damping under geometry assumptions
- A differential equation for modeling Nesterov's accelerated gradient method: theory and insights
- A dynamical view of nonlinear conjugate gradient methods with applications to FFT-based computational micromechanics
- A Lyapunov analysis of accelerated methods in optimization
- Discrete versus continuous Newton's method: A case study
- Fast convex optimization via inertial dynamics with Hessian driven damping
- scientific article; zbMATH DE number 3313108 (Why is no real title available?)
- Understanding the acceleration phenomenon via high-resolution differential equations
This page was built for publication: Analysis of continuous dynamical system models with Hessians derived from optimization methods
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6601496)