Higher-Order Corrections to Optimisers based on Newton's Method

arXiv2307.03820MaRDI QIDQ6443009FDOQ6443009

Authors: S. P. Brooks 

Publication date: 7 July 2023

Abstract: The Newton, Gauss--Newton and Levenberg--Marquardt methods all use the first derivative of a vector function (the Jacobian) to minimise its sum of squares. When the Jacobian matrix is ill-conditioned, the function varies much faster in some directions than others and the space of possible improvement in sum of squares becomes a long narrow ellipsoid in the linear model. This means that even a small amount of nonlinearity in the problem parameters can cause a proposed point far down the long axis of the ellipsoid to fall outside of the actual curved valley of improved values, even though it is quite nearby. This paper presents a differential equation that `follows' these valleys, based on the technique of geodesic acceleration, which itself provides a 2${}^{m}athrmnd$ order improvement to the Levenberg--Marquardt iteration step. Higher derivatives of this equation are computed that allow $n^{m}athrmth$ order improvements to the optimisation methods to be derived. These higher-order accelerated methods up to 4${}^{m}athrmth$ order are tested numerically and shown to provide substantial reduction of both number of steps and computation time.

This page was built for publication: Higher-Order Corrections to Optimisers based on Newton's Method

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6443009)