Trust Region Algorithms and Timestep Selection
error analysisfrust region algorithmglobal convergencegradient systemimplicit Euler methodLevenberg-Marquardt methodlocal convergencequadratic convergencesuperlinear convergencetimestep selectionunconstrained optimization
Nonlinear ordinary differential equations and systems (34A34) Numerical mathematical programming methods (65K05) Numerical methods for initial value problems involving ordinary differential equations (65L05) Multistep, Runge-Kutta and extrapolation methods for ordinary differential equations (65L06) Mesh generation, refinement, and adaptive methods for ordinary differential equations (65L50) Error bounds for numerical methods for ordinary differential equations (65L70) Nonlinear programming (90C30)
Unconstrained optimization problems are closely related to systems of ordinary differential equations (ODEs) with gradient structure. In this work, the author proves results that apply to both areas. He analyzes the convergence properties of a trust region, or Levenberg-Marquardt, algorithm for optimization. The algorithm may also be regarded as a linearized implicit Euler method with adaptive timestep for gradient ODEs. From the optimization viewpoint, the algorithm is driven directly by the Levenberg-Marquardt parameter rather than the trust region radius. The author gives a rigorous error analysis for the algorithm establishing global convergence and an unusual, extremely rapid, type of superlinear convergence. It is also shown how an inexpensive change to the algorithm leads to quadratic convergence. From the ODE viewpoint, the work contributes to the theory of gradient stability by presenting an algorithm that reproduces the correct global dynamics and gives very rapid local convergence to a stable steady state.
- An ODE-based trust region method for unconstrained optimization problems
- scientific article; zbMATH DE number 721738
- A modified ODE-based algorithm for unconstrained optimization problems
- scientific article; zbMATH DE number 1131709
- Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization
- Combining trust-region techniques and Rosenbrock methods to compute stationary points
- Computation of a trust region step
- Explicit pseudo-transient continuation and the trust-region updating strategy for unconstrained optimization
- Generalized continuation Newton methods and the trust-region updating strategy for the underdetermined system
- Continuation Newton methods with the residual trust-region time-stepping scheme for nonlinear equations
- Continuation methods with the trusty time-stepping scheme for linearly constrained optimization with noisy data
- The regularization continuation method with an adaptive time step control for linearly constrained optimization problems
- A comparison of methods for traversing regions of non-convexity in optimization problems
- A modified ODE-based algorithm for unconstrained optimization problems
- A method based on Rayleigh quotient gradient flow for extreme and interior eigenvalue problems
- Computational aerodynamic optimisation of vertical axis wind turbine blades
- The convergence of subspace trust region methods
- A new trust region method for unconstrained optimization
- An ODE-based nonmonotone method for unconstrained optimization problems
- Global convergence of a curvilinear search for non-convex optimization
- A nonmonotone ODE-based method for unconstrained optimization
- An ODE-like nonmonotone method for nonsmooth convex optimization
- Computing a Trust Region Step
- A Geometric Newton Method for Oja's Vector Field
- scientific article; zbMATH DE number 2059755 (Why is no real title available?)
- Primal-dual path-following methods and the trust-region updating strategy for linear programming with noisy data
- A new self-adaptive trust region method for unconstrained optimization
- Numerical methods for nonlinear equations
- Deep learning: an introduction for applied mathematicians
- Convergence analysis of the Levenberg–Marquardt method
- Analysis of a quasicontinuum method in one dimension
- Pseudotransient continuation for solving systems of nonsmooth equations with inequality constraints
- A hybrid BB-type method for solving large scale unconstrained optimization
- The regularization continuation method for optimization problems with nonlinear equality constraints
- Nonmonotone adaptive trust region method
- Backward error analysis and the qualitative behaviour of stochastic optimization algorithms: application to stochastic coordinate descent
- Continuation Newton methods with deflation techniques for global optimization problems
- A hybrid ODE-based method for unconstrained optimization problems
- Parallel pseudo-transient Newton-Krylov-Schwarz continuation algorithms for bifurcation analysis of incompressible sudden expansion flows
- A second-order pseudo-transient method for steady-state problems
This page was built for publication: Trust Region Algorithms and Timestep Selection
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4699110)
