Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution

DOI10.1016/J.JDE.2021.12.005zbMATH Open1489.37113arXiv2108.00203OpenAlexW3191545112MaRDI QIDQ2069167FDOQ2069167

H. Riahi, Z. Chbani, Hédy Attouch, Aicha Balhag

Publication date: 20 January 2022

Published in: Journal of Differential Equations (Search for Journal in Brave)

Abstract: In a Hilbert space, we provide a fast dynamic approach to the hierarchical minimization problem which consists in finding the minimum norm solution of a convex minimization problem. For this, we study the convergence properties of the trajectories generated by a damped inertial dynamic with Tikhonov regularization. When the time goes to infinity, the Tikhonov regularization parameter is supposed to tend towards zero, not too fast, which is a key property to make the trajectories strongly converge towards the minimizer of $f$ of minimum norm. According to the structure of the heavy ball method for strongly convex functions, the viscous damping coefficient is proportional to the square root of the Tikhonov regularization parameter. Therefore, it also converges to zero, which will ensure rapid convergence of values. Precisely, under a proper tuning of these parameters, based on Lyapunov's analysis, we show that the trajectories strongly converge towards the minimizer of minimum norm, and we provide the convergence rate of the values. We show a trade off between the property of fast convergence of values, and the property of strong convergence towards the minimum norm solution. This study improves several previous works where this type of results was obtained under restrictive hypotheses.

Full work available at URL: https://arxiv.org/abs/2108.00203

zbMATH Keywords

convex optimizationhierarchical minimizationaccelerated gradient methodsdamped inertial dynamicsNesterov accelerated gradient methodTikhonov approximation

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Numerical optimization and variational techniques (65K10) Convex programming (90C25) Management decision making, including multiple objectives (90B50) Dynamical systems in optimization and economics (37N40) Applications of functional analysis in optimization, convex analysis, mathematical programming, economics (46N10)

Cites Work

• Convex analysis and monotone operator theory in Hilbert spaces
• Introductory lectures on convex optimization. A basic course.
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• A Parallel Splitting Method for Coupled Monotone Inclusions
• Title not available (Why is that?)
• Viscosity Solutions of Minimization Problems
• Coupling Forward-Backward with Penalty Schemes and Parallel Splitting for Constrained Variational Inequalities
• Some methods of speeding up the convergence of iteration methods
• Fast convex optimization via inertial dynamics with Hessian driven damping
• On the long time behavior of second order differential equations with asymptotically small dissipation
• On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
• Proximal Point Algorithm Controlled by a Slowly Vanishing Term: Applications to Hierarchical Minimization
• Asymptotic behavior of coupled dynamical systems with multiscale aspects
• Prox-Penalization and Splitting Methods for Constrained Variational Problems
• Coupling the proximal point algorithm with approximation methods
• A dynamical approach to convex minimization coupling approximation with the steepest descent method
• Inertial gradient-like dynamical system controlled by a stabilizing term
• Asymptotic selection of viscosity equilibria of semilinear evolution equations by the introduction of a slowly vanishing term
• Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization
• Forward-backward and Tseng's type penalty schemes for monotone inclusion problems
• Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
• Convergence and asymptotic stabilization for some damped hyperbolic equations with non-isolated equilibria
• Asymptotic control and stabilization of nonlinear oscillators with non-isolated equilibria
• On an asymptotically autonomous system with Tikhonov type regularizing term
• A convergence result for nonautonomous subgradient evolution equations and its application to the steepest descent exponential penalty trajectory in linear programming
• The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than \(1/k^2\)
• Asymptotic behavior of gradient-like dynamical systems involving inertia and multiscale aspects
• A strongly convergent primal-dual method for nonoverlapping domain decomposition
• Combining fast inertial dynamics for convex optimization with Tikhonov regularization
• Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity
• The Differential Inclusion Modeling FISTA Algorithm and Optimality of Convergence Rate in the Case b $\leq3$
• Tikhonov regularization of a second order dynamical system with Hessian driven damping
• Asymptotic for a second order evolution equation with damping and regularizing terms
• A Liapunov function approach to the stabilization of second order coupled systems
• Title not available (Why is that?)

Cited In (14)

• Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution
• Fast convergence of inertial multiobjective gradient-like systems with asymptotic vanishing damping
• A fast continuous time approach for non-smooth convex optimization using Tikhonov regularization technique
• A Nesterov type algorithm with double Tikhonov regularization: fast convergence of the function values and strong convergence to the minimal norm solution
• On the strong convergence of the trajectories of a Tikhonov regularized second order dynamical system with asymptotically vanishing damping
• Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian
• On the strong convergence of continuous Newton-like inertial dynamics with Tikhonov regularization for monotone inclusions
• Second order dynamics featuring Tikhonov regularization and time scaling
• Fast convergence rate of values with strong convergence of trajectories via inertial dynamics with Tikhonov regularization terms and asymptotically vanishing damping
• Fast convex optimization via differential equation with Hessian-driven damping and Tikhonov regularization
• The heavy ball method regularized by Tikhonov term. Simultaneous convergence of values and trajectories
• Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization
• Strong Convergence of Trajectories via Inertial Dynamics Combining Hessian-Driven Damping and Tikhonov Regularization for General Convex Minimizations
• Convergence of damped inertial dynamics governed by regularized maximally monotone operators

This page was built for publication: Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2069167)