Fast optimization via inertial dynamics with closed-loop damping

DOI10.4171/jems/1231zbMath1523.37103arXiv2008.02261OpenAlexW3047531400MaRDI QIDQ6172672

Ernö Robert Csetnek, Radu Ioan Boţ, Hedy Attouch

Publication date: 20 July 2023

Published in: Journal of the European Mathematical Society (JEMS) (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2008.02261

zbMATH Keywords

convergence ratesKurdyka-Łojasiewicz inequalitymaximally monotone operatorsHessian dampingclosed-loop dampingdamped inertial gradient systemsquasi-gradient systems

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Convex programming (90C25) Numerical optimization and variational techniques (65K10) Newton-type methods (49M15) Applications of functional analysis in optimization, convex analysis, mathematical programming, economics (46N10) Dynamical systems in optimization and economics (37N40)

Related Items

Inertial Newton algorithms avoiding strict saddle points ⋮ First order inertial optimization algorithms with threshold effects associated with dry friction ⋮ Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian ⋮ Continuous Newton-like Methods Featuring Inertia and Variable Mass ⋮ A second order primal-dual dynamical system for a convex-concave bilinear saddle point problem ⋮ A second-order adaptive Douglas-Rachford dynamic method for maximal \(\alpha\)-monotone operators ⋮ On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• An inertial forward-backward algorithm for the minimization of the sum of two nonconvex functions
• Fast convex optimization via inertial dynamics with Hessian driven damping
• Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces
• Proximal alternating linearized minimization for nonconvex and nonsmooth problems
• On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
• Nonlinear damped partial differential equations and their uniform discretizations
• Asymptotics for some semilinear hyperbolic equations with non-autonomous damping
• Convergence to equilibrium for solutions of an abstract wave equation with general damping function
• Convergence of global and bounded solutions of a second order gradient like system with nonlinear dissipation and analytic nonlinearity
• Nonlinear evolution equations - global behavior of solutions
• Asymptotic convergence of nonlinear contraction semigroups in Hilbert space
• Convergence of solutions to second-order gradient-like systems with analytic nonlinearities
• A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics.
• A note on the dynamics of an oscillator in the presence of strong friction.
• Introductory lectures on convex optimization. A basic course.
• Inertial forward-backward algorithms with perturbations: application to Tikhonov regularization
• Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators
• Calculus of the exponent of Kurdyka-Łojasiewicz inequality and its applications to linear convergence of first-order methods
• Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity
• Stability and gradient dynamical systems
• Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward-backward splitting, and regularized Gauss-Seidel methods
• Convergence to equilibrium of solutions of the backward Euler scheme for asymptotically autonomous second-order gradient-like systems
• Approaching nonsmooth nonconvex minimization through second-order proximal-gradient dynamical systems
• A primal-dual dynamical approach to structured convex minimization problems
• Convergence rates of an inertial gradient descent algorithm under growth and flatness conditions
• Understanding the acceleration phenomenon via high-resolution differential equations
• First-order optimization algorithms via inertial systems with Hessian driven damping
• First-order inertial algorithms involving dry friction damping
• Universal bounds for a class of second order evolution equations and applications
• Convergence of a relaxed inertial proximal algorithm for maximally monotone operators
• Tikhonov regularization of a second order dynamical system with Hessian driven damping
• An extension of the second order dynamical system that models Nesterov's convex gradient method
• Accelerated proximal point method for maximally monotone operators
• Newton-like dynamics and forward-backward methods for structured monotone inclusions in Hilbert spaces
• Convergence rate of inertial forward-backward algorithm beyond Nesterov's rule
• On damped second-order gradient systems
• Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
• Every ordinary differential equation with a strict Lyapunov function is a gradient system
• Asymptotics for some nonlinear damped wave equation: finite time convergence versus exponential decay results
• Regular and singular pulse and front solutions and possible isochronous behavior in the short-pulse equation: phase-plane, multi-infinite series and variational approaches
• Second Order Forward-Backward Dynamical Systems For Monotone Inclusion Problems
• A dynamic approach to a proximal-Newton method for monotone inclusions in Hilbert spaces, with complexity O(1/n^2)
• The Rate of Convergence of Nesterov's Accelerated Forward-Backward Method is Actually Faster Than $1/k^2$
• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• Accelerated and Inexact Forward-Backward Algorithms
• iPiano: Inertial Proximal Algorithm for Nonconvex Optimization
• Variational Analysis in Sobolev and BV Spaces
• The Convergence Problem for Dissipative Autonomous Systems
• A Continuous Dynamical Newton-Like Approach to Solving Monotone Inclusions
• Finite Time Extinction by Nonlinear Damping for the Schrödinger Equation
• Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Łojasiewicz Inequality
• Asymptotic behavior of second-order dissipative evolution equations combining potential with non-potential effects
• Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity
• On the long time behavior of second order differential equations with asymptotically small dissipation
• APPLICATIONS OF THE ŁOJASIEWICZ–SIMON, GRADIENT INEQUALITY TO GRADIENT-LIKE EVOLUTION EQUATIONS
• An Invitation to Tame Optimization
• Convergence and asymptotic stabilization for some damped hyperbolic equations with non-isolated equilibria
• On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces
• THE HEAVY BALL WITH FRICTION METHOD, I. THE CONTINUOUS DYNAMICAL SYSTEM: GLOBAL EXPLORATION OF THE LOCAL MINIMA OF A REAL-VALUED FUNCTION BY ASYMPTOTIC ANALYSIS OF A DISSIPATIVE DYNAMICAL SYSTEM
• A forward-backward dynamical approach to the minimization of the sum of a nonsmooth convex with a smooth nonconvex function
• Convergence Rates of Inertial Forward-Backward Algorithms
• A generic online acceleration scheme for optimization algorithms via relaxation and inertia
• Asymptotic for a second-order evolution equation with convex potential andvanishing damping term
• Convergence Rates of the Heavy Ball Method for Quasi-strongly Convex Optimization
• Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3
• Finite Convergence of Proximal-Gradient Inertial Algorithms Combining Dry Friction with Hessian-Driven Damping
• Newton-like Inertial Dynamics and Proximal Algorithms Governed by Maximally Monotone Operators
• An Inertial Newton Algorithm for Deep Learning
• Weak convergence of the sequence of successive approximations for nonexpansive mappings
• Convex analysis and monotone operator theory in Hilbert spaces
• An inertial proximal method for maximal monotone operators via discretization of a nonlinear oscillator with damping