Certifying the Absence of Spurious Local Minima at Infinity
From MaRDI portal
Abstract: When searching for global optima of nonconvex unconstrained optimization problems, it is desirable that every local minimum be a global minimum. This property of having no spurious local minima is true in various problems of interest nowadays, including principal component analysis, matrix sensing, and linear neural networks. However, since these problems are non-coercive, they may yet have spurious local minima at infinity. The classical tools used to analyze the optimization landscape, namely the gradient and the Hessian, are incapable of detecting spurious local minima at infinity. In this paper, we identify conditions that certify the absence of spurious local minima at infinity, one of which is having bounded subgradient trajectories. We check that they hold in several applications of interest.
Recommendations
- Nonsmooth rank-one matrix factorization landscape
- Sharp restricted isometry bounds for the inexistence of spurious local minima in nonconvex matrix recovery
- Absence of local minima for a class of nonconvex optimization problems
- Role of sparsity and structure in the optimization landscape of non-convex matrix sensing
- Optimization Landscape of Neural Networks
Cites work
- \{Euclidean, metric, and Wasserstein\} gradient flows: an overview
- A Short Proof of the Variational Principle for Approximate Solutions of a Minimization Problem
- Clarke Subgradients of Stratifiable Functions
- Conservative set valued fields, automatic differentiation, stochastic gradient methods and deep learning
- Curves of descent
- Genericity in polynomial optimization
- Global Optimality in Low-Rank Matrix Optimization
- Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
- scientific article; zbMATH DE number 3855514 (Why is no real title available?)
- scientific article; zbMATH DE number 46303 (Why is no real title available?)
- scientific article; zbMATH DE number 1017538 (Why is no real title available?)
- scientific article; zbMATH DE number 1160037 (Why is no real title available?)
- scientific article; zbMATH DE number 1461253 (Why is no real title available?)
- scientific article; zbMATH DE number 877671 (Why is no real title available?)
- scientific article; zbMATH DE number 5046597 (Why is no real title available?)
- scientific article; zbMATH DE number 3238721 (Why is no real title available?)
- Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
- Linear structure of functions with maximal Clarke subdifferential
- Lipschitz functions with maximal Clarke subdifferentials are generic
- Low-rank matrix recovery with composite optimization: good conditioning and rapid convergence
- Model completeness results for expansions of the ordered field of real numbers by restricted Pfaffian functions and the exponential function
- Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview
- Nonsmooth rank-one matrix factorization landscape
- On the benefit of width for neural networks: disappearance of basins
- On the complexity of robust PCA and \(\ell_1\)-norm low-rank matrix approximation
- On the variational principle
- Pathological subgradient dynamics
- Real analysis
- Spurious valleys in one-hidden-layer neural network optimization landscapes
- Stochastic subgradient method converges on tame functions
- Suboptimal Local Minima Exist for Wide Neural Networks with Smooth Activations
- The Ekeland variational principle for set-valued maps involving coderivatives
- Variational Analysis in Sobolev and BV Spaces
Cited in
(5)- Homogeneous polynomials and spurious local minima on the unit sphere
- Role of sparsity and structure in the optimization landscape of non-convex matrix sensing
- Nonsmooth rank-one matrix factorization landscape
- Global convergence of the gradient method for functions definable in o-minimal structures
- Convergence of the Momentum Method for Semialgebraic Functions with Locally Lipschitz Gradients
This page was built for publication: Certifying the Absence of Spurious Local Minima at Infinity
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6116236)
