Accelerated gradient descent methods with line search
From MaRDI portal
The authors consider the unconstrained minimization problem on \(\mathbb{R}^n\) with a twice differentiable objective function \(f: \mathbb{R}^n\to\mathbb{R}\). A class of gradient descent methods is based on the multiplication of the iteration step size by an appropriate accleration parameter. The step-size is computed by a line search procedure. The methods of this class are called accelerated gradient descent algorithms with the line search. In the further part of the paper, a special accelerated gradient descent method arising from the Newton method with the line search is proposed. The acceleration parameter of this method is obtained by replacing the Hessian by an appropriately generated diagonal matrix. Linear convergence of the proposed algorithm is proved for uniformly convex objective functions satisfying some additional conditions. Reported numerical results show that the proposed method produces better results than alternative methods known from the literature.
Recommendations
- Multiplicative parameters in gradient descent methods
- An acceleration of gradient descent algorithm with backtracking for unconstrained opti\-mi\-za\-tion
- scientific article; zbMATH DE number 1960973
- Acceleration of conjugate gradient algorithms for unconstrained optimization
- Accelerated multiple step-size methods for solving unconstrained optimization problems
Cites work
- R-linear convergence of the Barzilai and Borwein gradient method
- A class of gradient unconstrained minimization algorithms with adaptive stepsize
- Alternate minimization gradient method
- Alternate step gradient method*
- An acceleration of gradient descent algorithm with backtracking for unconstrained opti\-mi\-za\-tion
- An unconstrained optimization test functions collection
- Analysis of monotone gradient methods
- Convergence Conditions for Ascent Methods
- Convergence of line search methods for unconstrained optimization
- Convex Analysis
- Efficent line search algorithm for unconstrained optimization
- Function minimization by conjugate gradients
- Gradient Method with Retards and Generalizations
- scientific article; zbMATH DE number 3529352 (Why is no real title available?)
- scientific article; zbMATH DE number 3381785 (Why is no real title available?)
- Minimization of functions having Lipschitz continuous first partial derivatives
- Modified two-point stepsize gradient methods for unconstrained optimization
- On Steepest Descent
- On the asymptotic behaviour of some new gradient methods
- On the Barzilai and Borwein choice of steplength for the gradient method
- Optimization theory and methods. Nonlinear programming
- Preconditioned Barzilai-Borwein method for the numerical solution of partial differential equations
- The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem
- The conjugate gradient method in extremal problems
- Two-Point Step Size Gradient Methods
Cited in
(34)- A transformation of accelerated double step size method for unconstrained optimization
- Accelerated double direction method for solving unconstrained optimization problems
- Hybrid modification of accelerated double direction method
- New hybrid conjugate gradient and Broyden-Fletcher-Goldfarb-Shanno conjugate gradient methods
- A note on a multiplicative parameters gradient method
- Hybridization of accelerated gradient descent method
- Scalar correction method for finding least-squares solutions on Hilbert spaces and its applications
- Theory of functional connections applied to quadratic and nonlinear programming under equality constraints
- An accelerated minimal gradient method with momentum for strictly convex quadratic optimization
- A note on hybridization process applied on transformed double step size model
- A survey of gradient methods for solving nonlinear optimization
- A novel value for the parameter in the Dai-Liao-type conjugate gradient method
- Accelerated primal-dual gradient descent with linesearch for convex, nonconvex, and nonsmooth optimization problems
- An acceleration of gradient descent algorithm with backtracking for unconstrained opti\-mi\-za\-tion
- Signal recovery with convex constrained nonlinear monotone equations through conjugate gradient hybrid approach
- Accelerated Dai-Liao projection method for solving systems of monotone nonlinear equations with application to image deblurring
- Accelerated line-search and trust-region methods
- An accelerated subspace minimization three-term conjugate gradient algorithm for unconstrained optimization
- Initial improvement of the hybrid accelerated gradient descent process
- Multiplicative parameters in gradient descent methods
- An improved derivative-free method via double direction approach for solving systems of nonlinear equations
- Finding approximate local minima faster than gradient descent
- Modified matrix-free methods for solving system of nonlinear equations
- Hybridization rule applied on accelerated double step size optimization scheme
- Primal-dual accelerated gradient methods with small-dimensional relaxation oracle
- Accelerated multiple step-size methods for solving unconstrained optimization problems
- A family of three-term conjugate gradient projection methods with a restart procedure and their relaxed-inertial extensions for the constrained nonlinear pseudo-monotone equations with applications
- Heterogeneous Mediation Analysis on Epigenomic PTSD and Traumatic Stress in a Predominantly African American Cohort
- Directional gradient and curvature approximation via Legendre quadrature in unconstrained optimization
- An accelerated gradient descent method based on a non-monotone backtracking line search scheme for unconstrained optimization and image restoration problems.
- Improvements to steepest descent method for multi-objective optimization
- Effective nonmonotone trust region method based on a simple cubic model for unconstrained optimization problems
- Legendre polynomial based approximation of the Hessian for quasi-Newton optimization
- An accelerated double step size model in unconstrained optimization
This page was built for publication: Accelerated gradient descent methods with line search
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5961879)
