Switched diffusion processes for non-convex optimization and saddle points search

DOI10.1007/S11222-023-10315-2MaRDI QIDQ6089196zbMATH OpenOpenAlexFDO

Authors Lucas Journel, Pierre Monmarché

Publication date 17 November 2023

Published in Statistics and Computing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2303.13160

zbMATH Keywords

metastabilitystochastic optimisationMarkov-modulated processsaddle point searchswitched diffusion process

Mathematics Subject Classification ID

Computational methods for problems pertaining to statistics (62-08) Diffusion processes (60J60)

Abstract: We introduce and investigate stochastic processes designed to find local minimizers and saddle points of non-convex functions, exploring the landscape more efficiently than the standard noisy gradient descent. The processes switch between two behaviours, a noisy gradient descent and a noisy saddle point search. It is proven to be well-defined and to converge to a stationary distribution in the long time. Numerical experiments are provided on low-dimensional toy models and for Lennard-Jones clusters.

Recommendations

• A Newton-based method for nonconvex optimization with fast evasion of saddle points
• Stochastic optimization with momentum: convergence, fluctuations, and traps avoidance
• A Diffusion Approximation Theory of Momentum Stochastic Gradient Descent in Nonconvex Optimization
• On the Global Convergence of Randomized Coordinate Gradient Descent for Nonconvex Optimization
• Behavior of accelerated gradient methods near critical points of nonconvex functions

Cites work

• An iterative minimization formulation for saddle point search
• Asymptotics of the spectral gap with applications to the theory of simulated annealing
• Averaging principle of SDE with small diffusion: Moderate deviations
• Constrained overdamped Langevin dynamics for symmetric multimarginal optimal transportation
• Controllability implies mixing. I: Convergence in the total variation metric
• Convergence analysis of discrete high-index saddle dynamics
• Convergence and cycling in Walker-type saddle search algorithms
• Convergence of the kinetic annealing for general potentials
• Dynamical systems under random perturbations with fast switching and slow diffusion: hyperbolic equilibria and stable limit cycles
• Exact long time behavior of some regime switching stochastic processes
• Fokker-Planck-Kolmogorov equations
• Free energy computations. A mathematical perspective
• Large deviations for Markov-modulated diffusion processes with rapid switching
• Long time behavior of diffusions with Markov switching
• Minimum action method for the study of rare events
• Multidimensional diffusion processes.
• On unique ergodicity for degenerate diffusions
• Option pricing model based on a Markov-modulated diffusion with jumps
• Partial differential equations and stochastic methods in molecular dynamics
• Random Perturbations of Dynamical Systems
• Simplified gentlest ascent dynamics for saddle points in non-gradient systems
• Some remarks on the model of the extended gentlest ascent dynamics
• Tail of a linear diffusion with Markov switching
• The gentlest ascent dynamics
• Yet Another Look at Harris’ Ergodic Theorem for Markov Chains

This page was built for publication: Switched diffusion processes for non-convex optimization and saddle points search

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6089196)