A Geometric Structure of Acceleration and Its Role in Making Gradients Small Fast

arXiv2106.10439MaRDI QIDQ6370720FDOQ6370720

Chan-Woo Park, Ernest K. Ryu, Jong-Min Lee

Publication date: 19 June 2021

Abstract: Since Nesterov's seminal 1983 work, many accelerated first-order optimization methods have been proposed, but their analyses lacks a common unifying structure. In this work, we identify a geometric structure satisfied by a wide range of first-order accelerated methods. Using this geometric insight, we present several novel generalizations of accelerated methods. Most interesting among them is a method that reduces the squared gradient norm with $mathcalO(1/K^4)$ rate in the prox-grad setup, faster than the $mathcalO(1/K^3)$ rates of Nesterov's FGM or Kim and Fessler's FPGM-m.

This page was built for publication: A Geometric Structure of Acceleration and Its Role in Making Gradients Small Fast

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6370720)