First-order optimization algorithms via inertial systems with Hessian driven damping

DOI10.1007/s10107-020-01591-1zbMath1497.37121arXiv1907.10536OpenAlexW3102391325MaRDI QIDQ2133411

Hassan Riahi, Zaki Chbani, Hedy Attouch, Jalal Fadili

Publication date: 29 April 2022

Published in: Mathematical Programming. Series A. Series B (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1907.10536

zbMATH Keywords

time rescalingRavine methodNesterov accelerated gradient methodHessian driven dampinginertial optimization algorithms

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Convex programming (90C25) Numerical optimization and variational techniques (65K10) Management decision making, including multiple objectives (90B50) Applications of functional analysis in optimization, convex analysis, mathematical programming, economics (46N10) Simulation of dynamical systems (37M05) Dynamical systems in optimization and economics (37N40)

Related Items

First-order inertial algorithms involving dry friction damping ⋮ Asymptotic behavior of Newton-like inertial dynamics involving the sum of potential and nonpotential terms ⋮ Fast convergence of dynamical ADMM via time scaling of damped inertial dynamics ⋮ Global convergence of model function based Bregman proximal minimization algorithms ⋮ Potential Function-Based Framework for Minimizing Gradients in Convex and Min-Max Optimization ⋮ First-order frameworks for continuous Newton-like dynamics governed by maximally monotone operators ⋮ From the Ravine Method to the Nesterov Method and Vice Versa: A Dynamical System Perspective ⋮ Newton-type inertial algorithms for solving monotone equations Governed by sums of potential and nonpotential operators ⋮ Accelerated differential inclusion for convex optimization ⋮ Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping ⋮ Accelerated dynamics with dry friction via time scaling and averaging of doubly nonlinear evolution equations ⋮ Fast convergence of inertial dynamics with Hessian-driven damping under geometry assumptions ⋮ Fast continuous dynamics inside the graph of subdifferentials of nonsmooth convex functions ⋮ An ordinary differential equation for modeling Halpern fixed-point Algorithm ⋮ Strong Convergence of Trajectories via Inertial Dynamics Combining Hessian-Driven Damping and Tikhonov Regularization for General Convex Minimizations ⋮ A speed restart scheme for a dynamics with Hessian-driven damping ⋮ Practical perspectives on symplectic accelerated optimization ⋮ A modified Tseng splitting method with double inertial steps for solving monotone inclusion problems ⋮ Inertial Newton algorithms avoiding strict saddle points ⋮ First order inertial optimization algorithms with threshold effects associated with dry friction ⋮ Smoothing accelerated proximal gradient method with fast convergence rate for nonsmooth convex optimization beyond differentiability ⋮ The heavy ball method regularized by Tikhonov term. Simultaneous convergence of values and trajectories ⋮ Convergence of inertial dynamics driven by sums of potential and nonpotential operators with implicit Newton-like damping ⋮ Time rescaling of a primal-dual dynamical system with asymptotically vanishing damping ⋮ Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian ⋮ Fast continuous dynamics inside the graph of maximally monotone operators ⋮ Fast optimization via inertial dynamics with closed-loop damping ⋮ Fast convex optimization via a third-order in time evolution equation: TOGES-V an improved version of TOGES* ⋮ Optimal convergence rates for damped inertial gradient dynamics with flat geometries ⋮ Newton-like Inertial Dynamics and Proximal Algorithms Governed by Maximally Monotone Operators ⋮ Accelerated proximal point method for maximally monotone operators ⋮ Second-Order Dynamics with Hessian-Driven Damping for Linearly Constrained Convex Minimization ⋮ Search Direction Correction with Normalized Gradient Makes First-Order Methods Faster ⋮ Convergence rates for the heavy-ball continuous dynamics for non-convex optimization, under Polyak-Łojasiewicz condition ⋮ Convergence rates of damped inerial dynamics from multi-degree-of-freedom system ⋮ Generalized Momentum-Based Methods: A Hamiltonian Perspective ⋮ Fast convex optimization via a third-order in time evolution equation ⋮ On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping ⋮ A fast continuous time approach with time scaling for nonsmooth convex optimization

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• Fast convex optimization via inertial dynamics with Hessian driven damping
• Global convergence of a closed-loop regularized Newton method for solving monotone inclusions in Hilbert spaces
• Universal gradient methods for convex optimization problems
• On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
• A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics.
• Rate of convergence of inertial gradient dynamics with time-dependent viscous damping coefficient
• Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity
• Convergence rate of inertial forward-backward algorithm beyond Nesterov's rule
• Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
• The Rate of Convergence of Nesterov's Accelerated Forward-Backward Method is Actually Faster Than $1/k^2$
• A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights
• Stability of Over-Relaxations for the Forward-Backward Algorithm, Application to FISTA
• On the long time behavior of second order differential equations with asymptotically small dissipation
• On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces
• Convergence Rates of Inertial Forward-Backward Algorithms
• Asymptotic for a second-order evolution equation with convex potential andvanishing damping term
• A Dynamical Approach to an Inertial Forward-Backward Algorithm for Convex Minimization
• Rate of convergence of the Nesterov accelerated gradient method in the subcritical case α ≤ 3
• Fast Proximal Methods via Time Scaling of Damped Inertial Dynamics
• Some methods of speeding up the convergence of iteration methods
• An introduction to continuous optimization for imaging
• Convex analysis and monotone operator theory in Hilbert spaces