Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution

DOI10.1016/J.JDE.2021.12.005MaRDI QIDQ2069167zbMATH OpenOpenAlexFDO

Authors Z. Chbani, H. Riahi, Hédy Attouch, Aicha Balhag

Publication date 20 January 2022

Published in Journal of Differential Equations (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2108.00203

zbMATH Keywords

accelerated gradient methodsconvex optimizationdamped inertial dynamicshierarchical minimizationNesterov accelerated gradient methodTikhonov approximation

Mathematics Subject Classification ID

Dynamical systems in optimization and economics (37N40) Applications of functional analysis in optimization, convex analysis, mathematical programming, economics (46N10) Numerical mathematical programming methods (65K05) Numerical optimization and variational techniques (65K10) Management decision making, including multiple objectives (90B50) Convex programming (90C25)

Abstract: In a Hilbert space, we provide a fast dynamic approach to the hierarchical minimization problem which consists in finding the minimum norm solution of a convex minimization problem. For this, we study the convergence properties of the trajectories generated by a damped inertial dynamic with Tikhonov regularization. When the time goes to infinity, the Tikhonov regularization parameter is supposed to tend towards zero, not too fast, which is a key property to make the trajectories strongly converge towards the minimizer of $f$ of minimum norm. According to the structure of the heavy ball method for strongly convex functions, the viscous damping coefficient is proportional to the square root of the Tikhonov regularization parameter. Therefore, it also converges to zero, which will ensure rapid convergence of values. Precisely, under a proper tuning of these parameters, based on Lyapunov's analysis, we show that the trajectories strongly converge towards the minimizer of minimum norm, and we provide the convergence rate of the values. We show a trade off between the property of fast convergence of values, and the property of strong convergence towards the minimum norm solution. This study improves several previous works where this type of results was obtained under restrictive hypotheses.

Recommendations

• Combining fast inertial dynamics for convex optimization with Tikhonov regularization
• On the strong convergence of the trajectories of a Tikhonov regularized second order dynamical system with asymptotically vanishing damping
• On the convergence of a class of inertial dynamical systems with Tikhonov regularization
• Tikhonov regularization of a second order dynamical system with Hessian driven damping
• Tikhonov regularization of a perturbed heavy ball system with vanishing damping

Cites work

• A convergence result for nonautonomous subgradient evolution equations and its application to the steepest descent exponential penalty trajectory in linear programming
• A dynamical approach to convex minimization coupling approximation with the steepest descent method
• A Liapunov function approach to the stabilization of second order coupled systems
• A Lyapunov analysis of accelerated methods in optimization
• A parallel splitting method for coupled monotone inclusions
• A strongly convergent primal-dual method for nonoverlapping domain decomposition
• Asymptotic behavior of coupled dynamical systems with multiscale aspects
• Asymptotic behavior of gradient-like dynamical systems involving inertia and multiscale aspects
• Asymptotic control and stabilization of nonlinear oscillators with non-isolated equilibria
• Asymptotic for a second order evolution equation with damping and regularizing terms
• Asymptotic selection of viscosity equilibria of semilinear evolution equations by the introduction of a slowly vanishing term
• Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity
• Combining fast inertial dynamics for convex optimization with Tikhonov regularization
• Convergence and asymptotic stabilization for some damped hyperbolic equations with non-isolated equilibria
• Convex analysis and monotone operator theory in Hilbert spaces
• Coupling Forward-Backward with Penalty Schemes and Parallel Splitting for Constrained Variational Inequalities
• Coupling the proximal point algorithm with approximation methods
• Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
• Fast convex optimization via inertial dynamics with Hessian driven damping
• Forward-backward and Tseng's type penalty schemes for monotone inclusion problems
• scientific article; zbMATH DE number 3850830 (Why is no real title available?)
• scientific article; zbMATH DE number 3551792 (Why is no real title available?)
• scientific article; zbMATH DE number 3806227 (Why is no real title available?)
• scientific article; zbMATH DE number 3893867 (Why is no real title available?)
• scientific article; zbMATH DE number 3227378 (Why is no real title available?)
• scientific article; zbMATH DE number 3307153 (Why is no real title available?)
• Inertial gradient-like dynamical system controlled by a stabilizing term
• Introductory lectures on convex optimization. A basic course.
• On an asymptotically autonomous system with Tikhonov type regularizing term
• On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
• On the long time behavior of second order differential equations with asymptotically small dissipation
• Prox-penalization and splitting methods for constrained variational problems
• Proximal Point Algorithm Controlled by a Slowly Vanishing Term: Applications to Hierarchical Minimization
• Some methods of speeding up the convergence of iteration methods
• Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization
• The Differential Inclusion Modeling FISTA Algorithm and Optimality of Convergence Rate in the Case b $\leq3$
• The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than 1/k^2
• Tikhonov regularization of a second order dynamical system with Hessian driven damping
• Viscosity Solutions of Minimization Problems

Cited in

(30)

• Convergence of damped inertial dynamics governed by regularized maximally monotone operators
• On the convergence of a class of inertial dynamical systems with Tikhonov regularization
• Combining fast inertial dynamics for convex optimization with Tikhonov regularization
• Tikhonov regularization of a perturbed heavy ball system with vanishing damping
• On the strong convergence of the trajectories of a Tikhonov regularized second order dynamical system with asymptotically vanishing damping
• On the strong convergence of continuous Newton-like inertial dynamics with Tikhonov regularization for monotone inclusions
• Strong Convergence of Trajectories via Inertial Dynamics Combining Hessian-Driven Damping and Tikhonov Regularization for General Convex Minimizations
• A fast continuous time approach for non-smooth convex optimization using Tikhonov regularization technique
• The heavy ball method regularized by Tikhonov term. Simultaneous convergence of values and trajectories
• Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian
• Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution
• Fast convergence of inertial multiobjective gradient-like systems with asymptotic vanishing damping
• A Nesterov type algorithm with double Tikhonov regularization: fast convergence of the function values and strong convergence to the minimal norm solution
• Second order dynamics featuring Tikhonov regularization and time scaling
• Fast convergence rate of values with strong convergence of trajectories via inertial dynamics with Tikhonov regularization terms and asymptotically vanishing damping
• Fast convex optimization via differential equation with Hessian-driven damping and Tikhonov regularization
• Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization
• Strong convergence and fast rates for dynamical systems with Tikhonov regularization
• Strong convergence behavior of a second order dynamical system with Hessian-driven and vanishing damping
• Fast rates and strong convergence towards the minimum norm solution of dynamic trajectories with Tikhonov regularization for convex minimization under linear constraints
• Fast primal-dual algorithm with Tikhonov regularization for a linear equality constrained convex optimization problem
• A proximal-gradient inertial algorithm with Tikhonov regularization: strong convergence to the minimal norm solution
• Inertial dynamics with vanishing Tikhonov regularization for multiobjective optimization
• Strong and fast convergences for damping, scaling and Tikhonov regularization
• A general mixed-order primal-dual dynamical system with Tikhonov regularization
• On the convergence of an inertial proximal algorithm with a Tikhonov regularization term
• Fast convergence of primal-dual dynamical systems with implicit Hessian damping and Tikhonov regularization
• Fast continuous-time approach for non-smooth convex minimization combining a constant damping coefficient, time scaling coefficient and Tikhonov regularization
• Asymptotic behavior of the Arrow-Hurwicz differential system with Tikhonov regularization
• On the structural monotone inclusion of continuous second-order dynamics combining dry friction and Hessian-driven damping.

This page was built for publication: Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2069167)