Accelerated gradient descent methods with line search (Q5961879)

The authors consider the unconstrained minimization problem on \(\mathbb{R}^n\) with a twice differentiable objective function \(f: \mathbb{R}^n\to\mathbb{R}\). A class of gradient descent methods is based on the multiplication of the iteration step size by an appropriate accleration parameter. The step-size is computed by a line search procedure. The methods of this class are called accelerated gradient descent algorithms with the line search. In the further part of the paper, a special accelerated gradient descent method arising from the Newton method with the line search is proposed. The acceleration parameter of this method is obtained by replacing the Hessian by an appropriately generated diagonal matrix. Linear convergence of the proposed algorithm is proved for uniformly convex objective functions satisfying some additional conditions. Reported numerical results show that the proposed method produces better results than alternative methods known from the literature.