Modern Theory of Gradient-Based Optimization

DOI10.48550/arXiv.2606.29206MaRDI QIDQ7270458zbMATH OpenarXivFDO

Authors Bin Shi

Publication date 28 June 2026

Published in arXiv

Tldr We survey gradient-based optimization techniques with an emphasis on ODE perspectives and extensions into discrete Lyapunov analysis. We provide a unified framework for analyzing advanced optimization algorithms using these tools.

Full work available at URL https://doi.org/10.48550/arXiv.2606.29206

arXiv classification

math.CAmath.NAmath.OC

Cites work

• A differential equation for modeling Nesterov's accelerated gradient method: theory and insights
• A Lyapunov Analysis of Accelerated PDHG Algorithms
• A Preconditioner for Generalized Saddle Point Problems
• Accelerated Implicit GDA Schemes: Theoretical Guarantees and Application to Proximal Augmented Lagrangian Methods
• An introduction to continuous optimization for imaging
• DYNAMICAL, SYMPLECTIC AND STOCHASTIC PERSPECTIVES ON GRADIENT-BASED OPTIMIZATION
• Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems
• Gradient Norm Minimization of Nesterov Acceleration: $o(1/k^3)$
• Lectures on convex optimization
• Linear convergence of forward-backward accelerated algorithms without knowledge of the modulus of strong convexity
• Linear Convergence of ISTA and FISTA
• Modern Theory of Gradient-Based Optimization
• Numerical solution of saddle point problems by block {Gram--Schmidt} orthogonalization
• On Underdamped Nesterov's Acceleration
• Proximal Subgradient Norm Minimization of ISTA and FISTA
• Some methods of speeding up the convergence of iteration methods
• Understanding the acceleration phenomenon via high-resolution differential equations
• Understanding the ADMM Algorithm via High-Resolution Differential Equations
• Understanding the PDHG Algorithm via High-Resolution Differential Equations

Cited in

(1)

• Modern Theory of Gradient-Based Optimization

This page was built for publication: Modern Theory of Gradient-Based Optimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q7270458)