Fast convex optimization via inertial dynamics with Hessian driven damping

DOI10.1016/j.jde.2016.08.020zbMath1375.49028arXiv1601.07113OpenAlexW2964216752MaRDI QIDQ324582

Hedy Attouch, Patrick Redont, Juan Peypouquet

Publication date: 17 October 2016

Published in: Journal of Differential Equations (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1601.07113

zbMATH Keywords

convex optimizationgradient flowsforward-backward algorithmfast convergent methodsHessian-driven dampinginertial dynamics

Mathematics Subject Classification ID

Newton-type methods (49M15) Methods involving semicontinuity and convergence; relaxation (49J45) Optimality conditions for problems involving ordinary differential equations (49K15)

Related Items (62)

Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling ⋮ Continuous dynamics related to monotone inclusions and non-smooth optimization problems ⋮ Accelerated methods with fastly vanishing subgradients for structured non-smooth minimization ⋮ First-order optimization algorithms via inertial systems with Hessian driven damping ⋮ Asymptotic behavior of Newton-like inertial dynamics involving the sum of potential and nonpotential terms ⋮ Fast convergence of dynamical ADMM via time scaling of damped inertial dynamics ⋮ The rate of convergence of optimization algorithms obtained via discretizations of heavy ball dynamical systems for convex optimization problems ⋮ Approaching nonsmooth nonconvex optimization problems through first order dynamical systems with hidden acceleration and Hessian driven damping terms ⋮ First-order frameworks for continuous Newton-like dynamics governed by maximally monotone operators ⋮ From the Ravine Method to the Nesterov Method and Vice Versa: A Dynamical System Perspective ⋮ A new minimizing-movements scheme for curves of maximal slope ⋮ Newton-type inertial algorithms for solving monotone equations Governed by sums of potential and nonpotential operators ⋮ Convergence of iterates for first-order optimization algorithms with inertia and Hessian driven damping ⋮ On the strong convergence of the trajectories of a Tikhonov regularized second order dynamical system with asymptotically vanishing damping ⋮ Accelerated dynamics with dry friction via time scaling and averaging of doubly nonlinear evolution equations ⋮ Unnamed Item ⋮ Fast convergence of inertial dynamics with Hessian-driven damping under geometry assumptions ⋮ On the strong convergence of continuous Newton-like inertial dynamics with Tikhonov regularization for monotone inclusions ⋮ Fast continuous dynamics inside the graph of subdifferentials of nonsmooth convex functions ⋮ On inexact relative-error hybrid proximal extragradient, forward-backward and Tseng's modified forward-backward methods with inertial effects ⋮ A speed restart scheme for a dynamics with Hessian-driven damping ⋮ Unnamed Item ⋮ Fast Krasnosel’skiĭ–Mann Algorithm with a Convergence Rate of the Fixed Point Iteration of \(\boldsymbol{{ o} \left(\frac{1}{{ k}} \right)}\) ⋮ A Systematic Approach to Lyapunov Analyses of Continuous-Time Models in Convex Optimization ⋮ Inertial Newton algorithms avoiding strict saddle points ⋮ First order inertial optimization algorithms with threshold effects associated with dry friction ⋮ A fast continuous time approach for non-smooth convex optimization using Tikhonov regularization technique ⋮ Convergence of inertial dynamics driven by sums of potential and nonpotential operators with implicit Newton-like damping ⋮ Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian ⋮ Fast continuous dynamics inside the graph of maximally monotone operators ⋮ Fast optimization via inertial dynamics with closed-loop damping ⋮ Continuous Newton-like Methods Featuring Inertia and Variable Mass ⋮ Fast convex optimization via a third-order in time evolution equation: TOGES-V an improved version of TOGES* ⋮ Second order splitting dynamics with vanishing damping for additively structured monotone inclusions ⋮ An Optimal First Order Method Based on Optimal Quadratic Averaging ⋮ Finite Convergence of Proximal-Gradient Inertial Algorithms Combining Dry Friction with Hessian-Driven Damping ⋮ Activity Identification and Local Linear Convergence of Forward--Backward-type Methods ⋮ Newton-like Inertial Dynamics and Proximal Algorithms Governed by Maximally Monotone Operators ⋮ Tikhonov regularization of a second order dynamical system with Hessian driven damping ⋮ Improved convergence rates and trajectory convergence for primal-dual dynamical systems with vanishing damping ⋮ An extension of the second order dynamical system that models Nesterov's convex gradient method ⋮ Convergence of inertial dynamics and proximal algorithms governed by maximally monotone operators ⋮ Convergence rates for an inertial algorithm of gradient type associated to a smooth non-convex minimization ⋮ Analysis of Optimization Algorithms via Integral Quadratic Constraints: Nonstrongly Convex Problems ⋮ A second-order dynamical approach with variable damping to nonconvex smooth minimization ⋮ On the convergence of a class of inertial dynamical systems with Tikhonov regularization ⋮ Continuous Newton-like inertial dynamics for monotone inclusions ⋮ Dynamical Systems Coupled with Monotone Set-Valued Operators: Formalisms, Applications, Well-Posedness, and Stability ⋮ Relative-error inertial-relaxed inexact versions of Douglas-Rachford and ADMM splitting algorithms ⋮ The Rate of Convergence of Nesterov's Accelerated Forward-Backward Method is Actually Faster Than $1/k^2$ ⋮ A second-order dynamical system with Hessian-driven damping and penalty term associated to variational inequalities ⋮ Fast Proximal Methods via Time Scaling of Damped Inertial Dynamics ⋮ Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution ⋮ A piecewise conservative method for unconstrained convex optimization ⋮ Understanding the acceleration phenomenon via high-resolution differential equations ⋮ A control-theoretic perspective on optimal high-order optimization ⋮ Time-varying continuous-time optimisation with pre-defined finite-time stability ⋮ Tikhonov Regularization of a Perturbed Heavy Ball System with Vanishing Damping ⋮ A primal-dual flow for affine constrained convex optimization ⋮ Fast convex optimization via a third-order in time evolution equation ⋮ On the effect of perturbations in first-order optimization methods with inertia and Hessian driven damping ⋮ A fast continuous time approach with time scaling for nonsmooth convex optimization

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Smooth minimization of non-smooth functions
• A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
• On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
• Asymptotic convergence of nonlinear contraction semigroups in Hilbert space
• A second-order gradient-like dissipative dynamical system with Hessian-driven damping. Application to optimization and mechanics.
• Introductory lectures on convex optimization. A basic course.
• Convex Optimization in Normed Spaces
• On the Long Time Behavior of Second Order Differential Equations with Asymptotically Small Dissipation
• On the long time behavior of second order differential equations with asymptotically small dissipation
• On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces
• THE HEAVY BALL WITH FRICTION METHOD, I. THE CONTINUOUS DYNAMICAL SYSTEM: GLOBAL EXPLORATION OF THE LOCAL MINIMA OF A REAL-VALUED FUNCTION BY ASYMPTOTIC ANALYSIS OF A DISSIPATIVE DYNAMICAL SYSTEM
• A Dynamical Approach to an Inertial Forward-Backward Algorithm for Convex Minimization
• Weak convergence of the sequence of successive approximations for nonexpansive mappings
• Convex analysis and monotone operator theory in Hilbert spaces

This page was built for publication: Fast convex optimization via inertial dynamics with Hessian driven damping