Gradient methods with memory

DOI10.1080/10556788.2020.1858831MaRDI QIDQ5043847zbMATH OpenOpenAlexFDO

Authors Mihai I. Florea, Yuri Nesterov

Publication date 6 October 2022

Published in Optimization Methods \& Software (Search for Journal in Brave)

Copyright license Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International

Full work available at URL https://arxiv.org/abs/2105.09241

zbMATH Keywords

convex optimizationgradient methodspiece-wise linear modelrate of convergencerelative smoothness

Mathematics Subject Classification ID

Convex programming (90C25) Methods of reduced gradient type (90C52)

Abstract: In this paper, we consider gradient methods for minimizing smooth convex functions, which employ the information obtained at the previous iterations in order to accelerate the convergence towards the optimal solution. This information is used in the form of a piece-wise linear model of the objective function, which provides us with much better prediction abilities as compared with the standard linear model. To the best of our knowledge, this approach was never really applied in Convex Minimization to differentiable functions in view of the high complexity of the corresponding auxiliary problems. However, we show that all necessary computations can be done very efficiently. Consequently, we get new optimization methods, which are better than the usual Gradient Methods both in the number of oracle calls and in the computational time. Our theoretical conclusions are confirmed by preliminary computational experiments.

Recommendations

• An optimal gradient method for smooth strongly convex minimization
• Gradient methods for minimizing composite functions
• Universal gradient methods for convex optimization problems
• A new memory gradient method and its convergence
• Exact gradient methods with memory

Cites work

• A descent lemma beyond Lipschitz gradient continuity: first-order methods revisited and applications
• An optimal first order method based on optimal quadratic averaging
• Complexity bounds for primal-dual methods minimizing the model of objective function
• Gradient methods for minimizing composite functions
• Lectures on convex optimization
• Relatively smooth convex optimization by first-order methods, and applications

Cited in

(11)

• Gradient methods for problems with inexact model of the objective
• Fast gradient descent for convex minimization problems with an oracle producing a ( , L)-model of function at the requested point
• Gradient methods with the exponential relaxation
• scientific article; zbMATH DE number 6001451 (Why is no real title available?)
• Universal gradient methods for convex optimization problems
• Exact gradient methods with memory
• Inexact model: a framework for optimization and variational inequalities
• Optimal Convergence Rates for the Proximal Bundle Method
• About some works of Boris Polyak on convergence of gradient methods and their development
• Dynamic smoothness parameter for fast gradient methods
• An optimal lower bound for smooth convex functions

This page was built for publication: Gradient methods with memory

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5043847)