Inexact Levenberg-Marquardt method for nonlinear equations (Q1884265)

The problem of solving a system of nonlinear equations \(F(x)=0\), where \(F:\mathbb{R}^m\to \mathbb{R}^n\) arises frequently from the discretisation of partial differential equations by finite differences, finite volumes, finite elements, boundary elements and other approximation methods. It is important in many applications in economy, nonlinear optimization and other areas. Let us assume that the set of all solutions \(X^*\) of this problems is not empty. The classical Levenberg-Marquardt (LM) method obtains the current trial step \(d_k\) by solving the system \((J^T_k J_k+ \mu_k I)\cdot d= -J^T_k F_k\), where \(F_k= F(x_k)\), \(J_k= F'(x_k)\) is the Jacobian, and \(\mu_k\geq 0\) is a parameter being updated from iteration to iteration. The LM step is a modification of the Newton's step \(d^N_k= -J^{-1}_k F_k\), when \(m= n\). The quadratic convergence of the LM method was proved if the \(d_k\) step is computed exactly and if the parameter \(\mu_k\) is chosen larger than a positive constant at each step, when the solution \(x^*\) is non-singular. But the exact computation of this step may be time consuming. In the paper it is shown that also if case of inexact computation of each step under a local error bound condition if the parameter \(\mu\) is chosen as \(\mu=\| F_k\|^2\) the sequence generated by the inexact LM method converges quadratically to some solution of the equation \(F(x)= 0\). The local error bound condition is satisfied everywhere if the Jacobian at the solution is non-singular, but these conditions are essential weaker that the non-singularity condition. The new inexact LM method, where the parameter is chosen to be large than a positive constant and the Jacobian at the solution is non-singular, preserve also a quadratic speed of convergence.