Escaping Saddle Points with the Successive Convex Approximation Algorithm
From MaRDI portal
Abstract: Optimizing non-convex functions is of primary importance in the vast majority of machine learning algorithms. Even though many gradient descent based algorithms have been studied, successive convex approximation based algorithms have been recently empirically shown to converge faster. However, such successive convex approximation based algorithms can get stuck in a first-order stationary point. To avoid that, we propose an algorithm that perturbs the optimization variable slightly at the appropriate iteration. In addition to achieving the same convergence rate results as the non-perturbed version, we show that the proposed algorithm converges to a second order stationary point. Thus, the proposed algorithm escapes the saddle point efficiently and does not get stuck at the first order saddle points.
This page was built for publication: Escaping Saddle Points with the Successive Convex Approximation Algorithm
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6315095)
