A new subspace minimization conjugate gradient method for unconstrained minimization

DOI10.1007/S10957-023-02325-XMaRDI QIDQ6150649zbMATH OpenFDO

Authors

Publication date 9 February 2024

Published in Journal of Optimization Theory and Applications (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2303.13345

zbMATH Keywords

conjugate gradient methodfinite termination propertyglobal convergencememoryless quasi-Newton methodsubspace minimization

Mathematics Subject Classification ID

Numerical methods for mathematical programming, optimization and variational techniques (65Kxx) Large-scale problems in mathematical programming (90C06)

Abstract: Subspace minimization conjugate gradient (SMCG) methods have become a class of quite efficient iterative methods for unconstrained optimization and have attracted extensive attention recently. Usually, the search directions of SMCG methods are generated by minimizing approximate models with the approximation matrix $B_k$ of the objective function at the current iterate over the subspace spanned by the current gradient $g_k$ and the latest search direction. The $g_k^T{B}_k{g}_k$ must be estimated properly in the calculation of the search directions, which is crucial to the theoretical properties and the numerical performance of SMCG methods. It is a great challenge to estimate it properly. The projection technique has been used successfully to generate conjugate gradient directions such as Dai-Kou conjugate gradient direction. Motivated by the above two observations, in the paper we present a new subspace minimization conjugate gradient methods by using a projection technique based on the memoryless quasi-Newton method. More specially, we project the search direction of the memoryless quasi-Newton method into the subspace spanned by the current gradient and the latest search direction and drive a new search direction, which is proved to be descent. Remarkably, the proposed method without any line search enjoys the finite termination property for two dimensional convex quadratic functions, which is helpful for designing algorithm. An adaptive scaling factor in the search direction is given based on the above finite termination property. The proposed method does not need to determine the parameter $h{o}_k$ and can be regarded as an extension of Dai-Kou conjugate gradient method. The global convergence of the proposed method is analyzed. Numerical comparisons indicate the proposed method is very promising.

Recommendations

• A subspace minimization conjugate gradient method based on conic model for unconstrained optimization
• A new subspace minimization conjugate gradient method based on modified secant equation for unconstrained optimization
• A new subspace minimization conjugate gradient method with nonmonotone line search for unconstrained optimization
• A class of accelerated subspace minimization conjugate gradient methods
• A Subspace Study on Conjugate Gradient Algorithms

Cites work

• A Barzilai-Borwein conjugate gradient method
• A class of accelerated subspace minimization conjugate gradient methods
• A Modified BFGS Algorithm for Unconstrained Optimization
• A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search
• A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search
• A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property
• A subspace conjugate gradient algorithm for large-scale unconstrained optimization
• A Subspace Study on Conjugate Gradient Algorithms
• Algorithm 851
• An accelerated subspace minimization three-term conjugate gradient algorithm for unconstrained optimization
• An efficient Barzilai-Borwein conjugate gradient method for unconstrained optimization
• An improved Dai-Kou conjugate gradient algorithm for unconstrained optimization
• Benchmarking optimization software with performance profiles.
• CUTEst: a constrained and unconstrained testing environment with safe threads for mathematical optimization
• Equipping the Barzilai-Borwein method with the two dimensional quadratic termination property
• Function minimization by conjugate gradients
• scientific article; zbMATH DE number 3843083 (Why is no real title available?)
• scientific article; zbMATH DE number 3278849 (Why is no real title available?)
• Linear convergence of subgradient algorithm for convex feasibility on Riemannian manifolds
• Methods of conjugate gradients for solving linear systems
• Modified two-point stepsize gradient methods for unconstrained optimization
• New conjugacy conditions and related nonlinear conjugate gradient methods
• New subspace minimization conjugate gradient methods based on regularization model for unconstrained optimization
• Nonlinear conjugate gradient methods for unconstrained optimization
• On the Convergence of a New Conjugate Gradient Algorithm
• Restart procedures for the conjugate gradient method
• Subgradient algorithms on Riemannian manifolds of lower bounded curvatures
• The conjugate gradient method in extremal problems

Cited in

(4)

• An efficient inertial subspace minimization CG algorithm with convergence rate analysis for constrained nonlinear monotone equations
• A globally convergent gradient method with momentum
• A limited memory subspace minimization conjugate gradient algorithm for unconstrained optimization
• Efficient globalization of heavy-ball type methods for unconstrained optimization based on curve searches

This page was built for publication: A new subspace minimization conjugate gradient method for unconstrained minimization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6150649)