Steepest descent preconditioning for nonlinear GMRES optimization
From MaRDI portal
Steepest descent preconditioning for nonlinear GMRES optimization
Abstract: Steepest descent preconditioning is considered for the recently proposed nonlinear generalized minimal residual (N-GMRES) optimization algorithm for unconstrained nonlinear optimization. Two steepest descent preconditioning variants are proposed. The first employs a line search, while the second employs a predefined small step. A simple global convergence proof is provided for the N-GMRES optimization algorithm with the first steepest descent preconditioner (with line search), under mild standard conditions on the objective function and the line search processes. Steepest descent preconditioning for N-GMRES optimization is also motivated by relating it to standard non-preconditioned GMRES for linear systems in the case of a quadratic optimization problem with symmetric positive definite operator. Numerical tests on a variety of model problems show that the N-GMRES optimization algorithm is able to very significantly accelerate convergence of stand-alone steepest descent optimization. Moreover, performance of steepest-descent preconditioned N-GMRES is shown to be competitive with standard nonlinear conjugate gradient and limited-memory Broyden-Fletcher-Goldfarb-Shanno methods for the model problems considered. These results serve to theoretically and numerically establish steepest-descent preconditioned N-GMRES as a general optimization method for unconstrained nonlinear optimization, with performance that appears promising compared to established techniques. In addition, it is argued that the real potential of the N-GMRES optimization framework lies in the fact that it can make use of problem-dependent nonlinear preconditioners that are more powerful than steepest descent (or, equivalently, N-GMRES can be used as a simple wrapper around any other iterative optimization process to seek acceleration of that process), and this potential is illustrated with a further application example.
Recommendations
- Steepest Descent and Conjugate Gradient Methods with Variable Preconditioning
- Preconditioned GMRES methods for least squares problems
- The steepest descent method with an adaptive alternating-triangular preconditioner
- Preconditioned steepest descent methods for some nonlinear elliptic equations involving p-Laplacian terms
- A preconditioned GMRES method
- A Geometric Convergence Theory for the Preconditioned Steepest Descent Iteration
- Toward efficient polynomial preconditioning for GMRES
- Nonlinear preconditioned conjugate gradient and least-squares finite elements
- A Preconditioned GMRES Method for Nonsymmetric or Indefinite Problems
- An optimally generalized steepest-descent algorithm for solving ill-posed linear systems
Cites work
- A Flexible Inner-Outer Preconditioned GMRES Algorithm
- A nonlinear GMRES optimization algorithm for canonical tensor decomposition
- A survey of nonlinear conjugate gradient methods
- Anderson acceleration for fixed-point iterations
- Extrapolation Methods for Vector Sequences
- Global Convergence Properties of Conjugate Gradient Methods for Optimization
- GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems
- scientific article; zbMATH DE number 1953444 (Why is no real title available?)
- Krylov subspace acceleration for nonlinear multigrid schemes
- Krylov Subspace Acceleration of Nonlinear Multigrid with Application to Recirculating Flows
- Line search algorithms with guaranteed sufficient decrease
- On multigrid for linear complementarity problems with application to American-style options
- Testing Unconstrained Optimization Software
- Two classes of multisecant methods for nonlinear acceleration
Cited in
(11)- Preconditioned steepest descent methods for some nonlinear elliptic equations involving p-Laplacian terms
- Rotational symmetry detection in 3D using reflectional symmetry candidates and quaternion-based rotation parameterization
- Nonlinearly preconditioned optimization on Grassmann manifolds for computing approximate Tucker tensor decompositions
- Composing scalable nonlinear algebraic solvers
- A nonlinearly preconditioned conjugate gradient algorithm for rank-R canonical tensor approximation.
- A Geometric Convergence Theory for the Preconditioned Steepest Descent Iteration
- On the asymptotic linear convergence speed of Anderson acceleration, Nesterov acceleration, and nonlinear GMRES
- Direct nonlinear acceleration
- Anderson acceleration as a Krylov method with application to convergence analysis
- Convergence properties of nonlinear GMRES applied to linear systems
- Some theoretical results on the finite convergence property and the temporary stalling behavior of Anderson acceleration on linear systems
This page was built for publication: Steepest descent preconditioning for nonlinear GMRES optimization
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2931517)
