A modification of the gradient method and function extremization (Q1263253)

The aim of this paper is to present a modification of the gradient method generating the sequence \(\{x_ k\}\) satisfying \(x_{k+1}=x_ k-\alpha \nabla f(x_ k),\) where \(\alpha\) is a constant such that \[ (*)\quad 0<\alpha <2/\| H(\bar x)\|_ s, \] \(H(\bar x)\) being the Hessian matrix of f at \(\bar x,\) \(\bar x\) a stationary point of the objective function f and \(\| H(\bar x)\|_ s\) the spectral norm of the matrix \(H(\bar x)\). Since \(\bar x\) is unknown, in general it is very difficult to choose \(\alpha\) so that (*) holds. Therefore the author proposes an iteration method \[ x_{k+1}=x_ k-(\beta /\| H(x_ k)\|)\nabla f(x_ k), \] with a constant \(\beta\) satisfying \(0<\beta <2\). It is proved that the sequence of points \(\{x_ k\}\) generated in this way converges to a minimum point of the function f under the same conditions assumed for the original gradient method. A numerical example illustrating the efficiency of the presented modification is given too.