Gradient Method with Retards and Generalizations
Let \({\mathbf A}\) be a real symmetric positive definite \(n \times n\) matrix and \(b\) be a real \(n\)-dimensional vector. The unique solution of the equation \({\mathbf A}x= b\) is the argument of the global minimum of the function \(f(x)\equiv \frac{1}{2} x^{T} {\mathbf A}x - b^{T}x\). Let us denote \(g(x)= {\mathbf A}x - b\) and let \(\lambda (x)\) be the minimizer of \(\varphi(\lambda) \equiv f(x - \lambda g(x))\). The minimum of \(f\) may be calculated using the classical steepest-descent iteration \(x_{k+1}=x_{k}-\lambda (x_{k})g(x_{k})\). The authors propose to use the iteration of the form \(x_{k+1}=x_{k}-\lambda (x_{\nu})g(x_{k})\) where \(\nu = \nu (k)\) is arbitrarily chosen in the set \(\{ k,k-1,\dots, \max \{ 0, k-m\}\}\). A practical motivation for the introduction of this iteration is that it allows to compute \(\lambda (x_{k})\) simultaneously in parallel with \(x_{k+1}\). For \(\nu (k)=k\) we obtain the steepest descent method, for \(\nu (k)= k-1 \) -- the iteration of \textit{J. Barzilai} and \textit{J. M. Borwein} [IMA J. Numer. Anal. 8, No. 1, 141-148 (1988; Zbl 0638.65055)]. The authors prove the global convergence of such iterations and discuss some strategies for the choice of \(\nu (k)\). In the convergence analysis, the authors consider the proposed iteration as a particular case of \(x_{k+1}=x_{k} - \frac {1}{\alpha _{k}} g(x_{k})\) where the scalar \(\alpha _{k}\) is the Rayleigh quotient of the matrix \({\mathbf A}\) at an \(n\)-dimensional vector, whose choice depends on the index \(k\). The proposed iteration was tested on two numerical problems, using 10 different strategies to choose the \(\nu (k)\).
- An affine-scaling interior-point CBB method for box-constrained optimization
- A descent algorithm without line search for unconstrained optimization
- Smooth and adaptive gradient method with retards
- A new adaptive Barzilai and Borwein method for unconstrained optimization
- A numerical study of applying spectral-step subgradient method for solving nonsmooth unconstrained optimization problems
- Stochastic gradient descent with Barzilai-Borwein update step for SVM
- An efficient gradient method with approximate optimal stepsize for large-scale unconstrained optimization
- Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming
- A dynamical Tikhonov regularization for solving ill-posed linear algebraic systems
- Linear convergence analysis of the use of gradient projection methods on total variation problems
- Iterative regularization algorithms for constrained image deblurring on graphics processors
- Properties of the delayed weighted gradient method
- On the inexact scaled gradient projection method
- On \(R\)-linear convergence analysis for a class of gradient methods
- Hybrid limited memory gradient projection methods for box-constrained optimization problems
- On the acceleration of the Barzilai-Borwein method
- Diagonal BFGS updates and applications to the limited memory BFGS method
- Fast gradient methods with alignment for symmetric linear systems without using Cauchy step
- Variable metric techniques for forward-backward methods in imaging
- New stepsizes for the gradient method
- A second-order gradient method for convex minimization
- A delayed weighted gradient method for strictly convex quadratic minimization
- A family of spectral gradient methods for optimization
- A flexible inexact-restoration method for constrained optimization
- Steepest descent method with random step lengths
- Synchrotron radiation-based \(l_1\)-norm regularization on micro-CT imaging in shale structure analysis
- Cooperative concurrent asynchronous computation of the solution of symmetric linear systems
- On the steplength selection in gradient methods for unconstrained optimization
- On the asymptotic behaviour of some new gradient methods
- New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds
- On memory gradient method with trust region for unconstrained optimization
- A new gradient method with an optimal stepsize property
- Faster gradient descent and the efficient recovery of images
- A globally optimal tri-vector method to solve an ill-posed linear system
- An alternating direction method for solving a class of inverse semi-definite quadratic programming problems
- Multiplier and gradient methods
- New iterative schemes for nonlinear fixed point problems, with applications to problems with bifurcations and incomplete-data problems
- A family of optimal weighted conjugate-gradient-type methods for strictly convex quadratic minimization
- On the regularizing behavior of the SDA and SDC gradient methods in the solution of linear ill-posed problems
- Algorithms that satisfy a stopping criterion, probably
- A harmonic framework for stepsize selection in gradient methods
- An optimal tri-vector iterative algorithm for solving ill-posed linear inverse problems
- Projected gradient algorithms for optimization over order simplices
- On the rate of convergence of projected Barzilai-Borwein methods
- scientific article; zbMATH DE number 4139497 (Why is no real title available?)
- A nonmonotone filter Barzilai-Borwein method for optimization
- scientific article; zbMATH DE number 16526 (Why is no real title available?)
- A Barzilai-Borwein-based heuristic algorithm for locating multiple facilities with regional demand
- Alternate minimization gradient method
- Alternate step gradient method*
- The chaotic nature of faster gradient descent methods
- An efficient gradient method with approximate optimal stepsize for the strictly convex quadratic minimization problem
- An efficient gradient method using the Yuan steplength
- Equipping the Barzilai-Borwein method with the two dimensional quadratic termination property
- Delayed gradient methods for symmetric and positive definite linear systems
- Error minimizing relaxation strategies in Landweber and Kaczmarz type iterations
- Spectral properties of Barzilai-Borwein rules in solving singly linearly constrained optimization problems subject to lower and upper bounds
- A new modified Barzilai-Borwein gradient method for the quadratic minimization problem
- Gradient methods exploiting spectral properties
- Gradient descent and fast artificial time integration
- Gradient projection methods for quadratic programs and applications in training support vector machines
- Secant Acceleration of Sequential Residual Methods for Solving Large-Scale Nonlinear Systems of Equations
- Accelerated gradient descent methods with line search
- Alternating cyclic vector extrapolation technique for accelerating nonlinear optimization algorithms and fixed-point mapping applications
- On the Preconditioned Delayed Weighted Gradient Method
- Geometric interpretation of some Cauchy related methods
- Delayed weighted gradient method with simultaneous step-sizes for strongly convex optimization
- New gradient methods with adaptive stepsizes by approximate models
- A hybrid spectral projection-based method for solving monotone nonlinear systems and signal recovery
- A cyclic delayed weighted steplength for the gradient method
- A note on R-linear convergence of nonmonotone gradient methods
- On a family of relaxed gradient descent methods for strictly convex quadratic minimization
- A limited memory steepest descent method
- Regularized Barzilai-Borwein method
- A speed up strategy for gradient methods
- Global and Unconditional R-Linear Convergence of Rank-Compressed Weighted LMSD Sweeps for Strictly Convex Quadratics
- R-Linear Convergence of Barzilai-Borwein Methods on Strictly Convex Quadratics via Closed-Cone Homogeneous Dynamics
- Accelerating gradient projection methods for \(\ell _1\)-constrained signal recovery by steplength selection rules
- Hybrid spectral gradient method for the unconstrained minimization problem
- Partial spectral projected gradient method with active-set strategy for linearly constrained optimization
- Gradient methods with adaptive step-sizes
- Notes on the Dai-Yuan-Yuan modified spectral gradient method
This page was built for publication: Gradient Method with Retards and Generalizations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4243561)
