Local Saddle Point Optimization: A Curvature Exploitation Approach
From MaRDI portal
Publication:6301591
arXiv1805.05751MaRDI QIDQ6301591FDOQ6301591
Thomas Hofmann, Aurelien Lucchi, Hadi Daneshmand, Leonard Adolphs
Publication date: 15 May 2018
Abstract: Gradient-based optimization methods are the most popular choice for finding local optima for classical minimization and saddle point problems. Here, we highlight a systemic issue of gradient dynamics that arise for saddle point problems, namely the presence of undesired stable stationary points that are no local optima. We propose a novel optimization approach that exploits curvature information in order to escape from these undesired stationary points. We prove that different optimization methods, including gradient method and Adagrad, equipped with curvature exploitation can escape non-optimal stationary points. We also provide empirical results on common saddle point problems which confirm the advantage of using curvature exploitation.
Has companion code repository: https://github.com/limcherhang/finalreport
This page was built for publication: Local Saddle Point Optimization: A Curvature Exploitation Approach
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6301591)
