Dynamic behavior for a gradient algorithm with energy and momentum

Abstract: This paper investigates a novel gradient algorithm, using both energy and momentum (called AGEM), for solving general non-convex optimization problems. The solution properties of the AGEM algorithm, including uniformly boundedness and convergence to critical points, are examined. The dynamic behavior is studied through analysis of a high-resolution ODE system. Such ODE system is nonlinear and obtained by taking the limit of the discrete scheme while keeping the momentum effect through a rescale of the momentum parameter. In particular, we show global well-posedness of the ODE system, time-asymptotic convergence of solution trajectories, and further establish a linear convergence rate for objective functions satisfying the Polyak-Lojasiewicz condition.

This page was built for publication: Dynamic behavior for a gradient algorithm with energy and momentum

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6507772)