Friction-adaptive descent: a family of dynamics-based optimization methods
From MaRDI portal
Publication:6087370
Hamiltonian systemgradient descentcubic dampingconvex and nonconvex optimizationNosé-Hoover dynamicsadaptive LangevinPolyak heavy ball method
Numerical optimization and variational techniques (65K10) Simulation of dynamical systems (37M05) Discretization methods and integrators (symplectic, variational, geometric, etc.) for dynamical systems (37M15) Computational molecular dynamics in statistical mechanics (82M37) Computational methods for attractors of dynamical systems (37M22)
Abstract: We describe a family of descent algorithms which generalizes common existing schemes used in applications such as neural network training and more broadly for optimization of smooth functions--potentially for global optimization, or as a local optimization method to be deployed within global optimization schemes like basin hopping. By introducing an auxiliary degree of freedom we create a dynamical system with improved stability, reducing oscillatory modes and accelerating convergence to minima. The resulting algorithms are simple to implement and control, and convergence can be shown directly by Lyapunov's second method. Although this framework, which we refer to as friction-adaptive descent (FAD), is fairly general, we focus most of our attention here on a specific variant: kinetic energy stabilization (which can be viewed as a zero-temperature Nos'e--Hoover scheme but with added dissipation in both physical and auxiliary variables), termed KFAD (kinetic FAD). To illustrate the flexibility of the FAD framework we consider several other methods. in certain asymptotic limits, these methods can be viewed as introducing cubic damping in various forms; they can be more efficient than linearly dissipated Hamiltonian dynamics in common optimization settings. We present details of the numerical methods and show convergence for both the continuous and discretized dynamics in the convex setting by constructing Lyapunov functions. The methods are tested using a toy model (the Rosenbrock function). We also demonstrate the methods for structural optimization for atomic clusters in Lennard--Jones and Morse potentials. The experiments show the relative efficiency and robustness of FAD in comparison to linearly dissipated Hamiltonian dynamics.
Recommendations
Cites work
- scientific article; zbMATH DE number 3159869 (Why is no real title available?)
- scientific article; zbMATH DE number 3773010 (Why is no real title available?)
- scientific article; zbMATH DE number 2107836 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- A Projective Thermostatting Dynamics Technique
- A Systematic Approach to Lyapunov Analyses of Continuous-Time Models in Convex Optimization
- A differential equation for modeling Nesterov's accelerated gradient method: theory and insights
- A general system of differential equations to model first-order adaptive algorithms
- A gentle stochastic thermostat for molecular dynamics
- Conformal Hamiltonian systems
- Deep learning
- Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.
- Exponential relaxation of the Nosé-Hoover thermostat under Brownian heating
- Hypocoercivity properties of adaptive Langevin dynamics
- Introductory lectures on convex optimization. A basic course.
- Langevin dynamics with general kinetic energies
- Rational construction of stochastic numerical methods for molecular sampling
- Second order conformal symplectic schemes for damped Hamiltonian systems
- Some methods of speeding up the convergence of iteration methods
- Splitting methods
This page was built for publication: Friction-adaptive descent: a family of dynamics-based optimization methods
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6087370)
