Gradient-Type Methods for Optimization Problems with Polyak-Łojasiewicz Condition: Early Stopping and Adaptivity to Inexactness Parameter

DOI10.1007/978-3-031-22990-9_2MaRDI QIDQ6090757zbMATH OpenOpenAlexFDO

Authors Ilya A. Kuruzov, F. S. Stonyakin, Mohammad S. Alkousa

Publication date 17 November 2023

Published in Communications in Computer and Information Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2212.04226

zbMATH Keywords

adaptive methodgradient methodinexact gradientPolyak-Łojasiewicz condition

Mathematics Subject Classification ID

Nonlinear programming (90C30) Applications of mathematical programming (90C90)

Abstract: Due to its applications in many different places in machine learning and other connected engineering applications, the problem of minimization of a smooth function that satisfies the Polyak-{L}ojasiewicz condition receives much attention from researchers. Recently, for this problem, the authors of recent work proposed an adaptive gradient-type method using an inexact gradient. The adaptivity took place only with respect to the Lipschitz constant of the gradient. In this paper, for problems with the Polyak-{L}ojasiewicz condition, we propose a full adaptive algorithm, which means that the adaptivity takes place with respect to the Lipschitz constant of the gradient and the level of the noise in the gradient. We provide a detailed analysis of the convergence of the proposed algorithm and an estimation of the distance from the starting point to the output point of the algorithm. Numerical experiments and comparisons are presented to illustrate the advantages of the proposed algorithm in some examples.

Recommendations

• An optimal extension of the Polak-Ribière-Polyak conjugate gradient method
• An extension of a theorem by B. T. Polyak on gradient-type methods
• Gradient methods for problems with inexact model of the objective
• A modified sufficient descent Polak-Ribiére-Polyak type conjugate gradient method for unconstrained optimization problems
• Gradient methods with regularization for constrained optimization problems and their complexity estimates
• An extragradient-like approximation method for variational inequalities and fixed point problems
• The global convergence of the Polak-Ribière-Polyak conjugate gradient algorithm under inexact line search for nonconvex functions
• An extragradient-like approximation method for variational inequality problems and fixed point problems
• scientific article; zbMATH DE number 1266985
• A modified Polak-Ribière-Polyak conjugate gradient algorithm for nonsmooth convex programs

Cites work

• A geometric analysis of phase retrieval
• First-order methods in optimization
• First-order methods of smooth convex optimization with inexact oracle
• Global optimization with non-convex constraints. Sequential and parallel algorithms
• scientific article; zbMATH DE number 3791120 (Why is no real title available?)
• Inexact model: a framework for optimization and variational inequalities
• Inverse and ill-posed problems. Theory and applications.
• Lectures on convex optimization
• Safe global optimization of expensive noisy black-box functions in the -Lipschitz framework
• Sequential subspace optimization for quasar-convex optimization problems with inexact gradient
• Smooth Optimization with Approximate Gradient
• Stopping rules for gradient methods for non-convex problems with additive noise in gradient
• Universal gradient methods for convex optimization problems

Cited in

(1)

• About some works of Boris Polyak on convergence of gradient methods and their development

This page was built for publication: Gradient-Type Methods for Optimization Problems with Polyak-Łojasiewicz Condition: Early Stopping and Adaptivity to Inexactness Parameter

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6090757)