Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution

DOI10.1016/J.JDE.2021.12.005MaRDI QIDQ2069167zbMATH OpenOpenAlexFDO

Authors Z. Chbani, H. Riahi, Hédy Attouch, Aicha Balhag

Publication date 20 January 2022

Published in Journal of Differential Equations (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2108.00203

zbMATH Keywords

convex optimizationhierarchical minimizationaccelerated gradient methodsdamped inertial dynamicsNesterov accelerated gradient methodTikhonov approximation

Mathematics Subject Classification ID

Numerical mathematical programming methods (65K05) Numerical optimization and variational techniques (65K10) Convex programming (90C25) Management decision making, including multiple objectives (90B50) Dynamical systems in optimization and economics (37N40) Applications of functional analysis in optimization, convex analysis, mathematical programming, economics (46N10)

Abstract: In a Hilbert space, we provide a fast dynamic approach to the hierarchical minimization problem which consists in finding the minimum norm solution of a convex minimization problem. For this, we study the convergence properties of the trajectories generated by a damped inertial dynamic with Tikhonov regularization. When the time goes to infinity, the Tikhonov regularization parameter is supposed to tend towards zero, not too fast, which is a key property to make the trajectories strongly converge towards the minimizer of $f$ of minimum norm. According to the structure of the heavy ball method for strongly convex functions, the viscous damping coefficient is proportional to the square root of the Tikhonov regularization parameter. Therefore, it also converges to zero, which will ensure rapid convergence of values. Precisely, under a proper tuning of these parameters, based on Lyapunov's analysis, we show that the trajectories strongly converge towards the minimizer of minimum norm, and we provide the convergence rate of the values. We show a trade off between the property of fast convergence of values, and the property of strong convergence towards the minimum norm solution. This study improves several previous works where this type of results was obtained under restrictive hypotheses.

Recommendations

• Combining fast inertial dynamics for convex optimization with Tikhonov regularization
• On the strong convergence of the trajectories of a Tikhonov regularized second order dynamical system with asymptotically vanishing damping
• On the convergence of a class of inertial dynamical systems with Tikhonov regularization
• Tikhonov regularization of a second order dynamical system with Hessian driven damping
• Tikhonov regularization of a perturbed heavy ball system with vanishing damping

Cites work

• scientific article; zbMATH DE number 3850830 (Why is no real title available?)
• scientific article; zbMATH DE number 3551792 (Why is no real title available?)
• scientific article; zbMATH DE number 3806227 (Why is no real title available?)
• scientific article; zbMATH DE number 7370630 (Why is no real title available?)
• scientific article; zbMATH DE number 3893867 (Why is no real title available?)
• scientific article; zbMATH DE number 3227378 (Why is no real title available?)
• scientific article; zbMATH DE number 3307153 (Why is no real title available?)
• A Liapunov function approach to the stabilization of second order coupled systems
• A convergence result for nonautonomous subgradient evolution equations and its application to the steepest descent exponential penalty trajectory in linear programming
• A dynamical approach to convex minimization coupling approximation with the steepest descent method
• A parallel splitting method for coupled monotone inclusions
• A strongly convergent primal-dual method for nonoverlapping domain decomposition
• Asymptotic behavior of coupled dynamical systems with multiscale aspects
• Asymptotic behavior of gradient-like dynamical systems involving inertia and multiscale aspects
• Asymptotic control and stabilization of nonlinear oscillators with non-isolated equilibria
• Asymptotic for a second order evolution equation with damping and regularizing terms
• Asymptotic selection of viscosity equilibria of semilinear evolution equations by the introduction of a slowly vanishing term
• Asymptotic stabilization of inertial gradient dynamics with time-dependent viscosity
• Combining fast inertial dynamics for convex optimization with Tikhonov regularization
• Convergence and asymptotic stabilization for some damped hyperbolic equations with non-isolated equilibria
• Convex analysis and monotone operator theory in Hilbert spaces
• Coupling Forward-Backward with Penalty Schemes and Parallel Splitting for Constrained Variational Inequalities
• Coupling the proximal point algorithm with approximation methods
• Fast convergence of inertial dynamics and algorithms with asymptotic vanishing viscosity
• Fast convex optimization via inertial dynamics with Hessian driven damping
• Forward-backward and Tseng's type penalty schemes for monotone inclusion problems
• Inertial gradient-like dynamical system controlled by a stabilizing term
• Introductory lectures on convex optimization. A basic course.
• On an asymptotically autonomous system with Tikhonov type regularizing term
• On the convergence of the iterates of the ``fast iterative shrinkage/thresholding algorithm
• On the long time behavior of second order differential equations with asymptotically small dissipation
• Prox-penalization and splitting methods for constrained variational problems
• Proximal Point Algorithm Controlled by a Slowly Vanishing Term: Applications to Hierarchical Minimization
• Some methods of speeding up the convergence of iteration methods
• Strong asymptotic convergence of evolution equations governed by maximal monotone operators with Tikhonov regularization
• The Differential Inclusion Modeling FISTA Algorithm and Optimality of Convergence Rate in the Case b $\leq3$
• The rate of convergence of Nesterov's accelerated forward-backward method is actually faster than \(1/k^2\)
• Tikhonov regularization of a second order dynamical system with Hessian driven damping
• Viscosity Solutions of Minimization Problems

Cited in

(17)

• On the convergence of a class of inertial dynamical systems with Tikhonov regularization
• Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution
• Fast convergence of inertial multiobjective gradient-like systems with asymptotic vanishing damping
• A fast continuous time approach for non-smooth convex optimization using Tikhonov regularization technique
• A Nesterov type algorithm with double Tikhonov regularization: fast convergence of the function values and strong convergence to the minimal norm solution
• On the strong convergence of the trajectories of a Tikhonov regularized second order dynamical system with asymptotically vanishing damping
• Combining fast inertial dynamics for convex optimization with Tikhonov regularization
• Accelerated gradient methods combining Tikhonov regularization with geometric damping driven by the Hessian
• On the strong convergence of continuous Newton-like inertial dynamics with Tikhonov regularization for monotone inclusions
• Second order dynamics featuring Tikhonov regularization and time scaling
• Fast convergence rate of values with strong convergence of trajectories via inertial dynamics with Tikhonov regularization terms and asymptotically vanishing damping
• Fast convex optimization via differential equation with Hessian-driven damping and Tikhonov regularization
• The heavy ball method regularized by Tikhonov term. Simultaneous convergence of values and trajectories
• Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization
• Tikhonov regularization of a perturbed heavy ball system with vanishing damping
• Convergence of damped inertial dynamics governed by regularized maximally monotone operators
• Strong Convergence of Trajectories via Inertial Dynamics Combining Hessian-Driven Damping and Tikhonov Regularization for General Convex Minimizations

This page was built for publication: Damped inertial dynamics with vanishing Tikhonov regularization: strong asymptotic convergence towards the minimum norm solution

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2069167)