Local analysis of a spectral correction for the Gauss-Newton model applied to quadratic residual problems (Q329315)

This paper is a helpful contribution to nonlinear programming at its interface with data mining, statistics and the theory of inverse problems, being a ``classical'' and yet challenging field of modern operational research (OR), but also of economics, medicine, engineering, etc. There, smart inventions are appreciated very much to cope with problems of complexity, of convergence and to further provide qualitative theory. In fact, in today's OR, the strongly emerging area of data mining is often named under ``big data'', ``data analytics'', ``business analytics'', or just ``analytics''. This paper is rigorous and based on it, future research can be raised and so many real-world applications made. In fact, the authors present a simple -- smart -- spectral correction for the famous Gauss-Newton model from nonlinear regression theory, namely, from the optimization of nonlinear least squares. That correction consists in the addition of a sign-free multiple of the unit (or identity) matrix to the Gauss-Newton model's Hessian, namely, the multiple that bases on spectral approximations for the Hessians of the residual functions. For the resulting method, an analysis of local convergence is provided in detail and applied on the class of quadratic residual problems. Given some mild assumptions, the proposed technique is demonstrated to converge for problems where convergence of the Gauss-Newton procedure might not be guaranteed. Furthermore, for a class of non-zero residue problems the rate of linear convergence is shown to be better than the one of the Gauss-Newton method. With the help of numerical examples on quadratic and non-quadratic residual problems, these theoretical results are illustrated in the article. This valuable article is well-structured, mathematically deep, well exemplified and illustrated, related to rank-deficiency, etc., also, and written well. The five sections of this work are as follows: 1. Introduction, 2. The spectral approximation, 3. Local analysis: quadratic residual case, 4. Illustrative numerical examples, and 5. Conclusions and future perspectives. In the future, theoretical and algorithmic refinements and generalizations, strong results and codes could be expected in the research community, initiated by this research article. Those could be made in terms of nonlinear optimization, semi-infinite optimization, robust optimization, optimal control, image processing, pattern recognition, shape detection, tomography and information Theory. Such a process could foster progress in science and engineering, finance, business administration, economics, earth-sciences, neuroscience and medicine.