Gradient descent provably escapes saddle points in the training of shallow ReLU networks
From MaRDI portal
(Redirected from Publication:6655804)
Gradient descent provably escapes saddle points in the training of shallow ReLU networks (scientific article; zbMATH DE number 7960846)
Gradient descent provably escapes saddle points in the training of shallow ReLU networks (scientific article; zbMATH DE number 7960846)
Recommendations
- Gradient descent optimizes over-parameterized deep ReLU networks
- The global optimization geometry of shallow linear neural networks
- Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions
- Non-differentiable saddle points and sub-optimal local minima exist for deep ReLU networks
Cites work
- A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
- A geometric analysis of phase retrieval
- A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions
- Behavior of accelerated gradient methods near critical points of nonconvex functions
- Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture
- Convergence of the Iterates of Descent Methods for Analytic Cost Functions
- First-order methods almost always avoid strict saddle points
- Gradient descent only converges to minimizers: non-isolated critical points and invariant regions
- Gradient descent optimizes over-parameterized deep ReLU networks
- scientific article; zbMATH DE number 3980052 (Why is no real title available?)
- Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions
- Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
- Measure theory and fine properties of functions
- Nonconvergence to unstable points in urn models and stochastic approximations
- Nonconvex Robust Low-Rank Matrix Recovery
- Splitting methods with variable metric for Kurdyka-Łojasiewicz functions and general convergence rates
- Spurious valleys in one-hidden-layer neural network optimization landscapes
- Stochastic subgradient method converges on tame functions
- The nonsmooth landscape of phase retrieval
- Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
Cited in
(5)- When does gradient descent with logistic loss find interpolating two-layer networks?
- Universality of gradient descent neural network training
- Limitations of neural network training due to numerical instability of backpropagation
- A deterministic gradient-based approach to avoid saddle points
- Kurdyka-Łojasiewicz exponent via Hadamard parametrization
This page was built for publication: Gradient descent provably escapes saddle points in the training of shallow ReLU networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6655804)
