Gradient descent provably escapes saddle points in the training of shallow ReLU networks
type: Item
instance of: scholarly article
title: Gradient descent provably escapes saddle points in the training of shallow ReLU networks
author: Patrick Cheridito
author: Arnulf Jentzen
author: Florian Rossmannek
published in: Journal of Optimization Theory and Applications
publication date: 2024-12-27T00:00:00Z
Mathematics Subject Classification ID: 68T07
Mathematics Subject Classification ID: 90C26
Mathematics Subject Classification ID: 65K10
zbMATH DE Number: 7960846
zbMATH Keywords: neural networks
zbMATH Keywords: center-stable manifolds
zbMATH Keywords: gradient descent
zbMATH Keywords: nonconvex optimization
MaRDI profile type: MaRDI publication profile
cites work: Convergence of the Iterates of Descent Methods for Analytic Cost Functions
cites work: Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
cites work: A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions
cites work: Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions
cites work: Stochastic subgradient method converges on tame functions
cites work: The nonsmooth landscape of phase retrieval
cites work: A comparative analysis of optimization and generalization properties of two-layer neural network and random feature models under gradient descent dynamics
cites work: Q5178046
cites work: Splitting methods with variable metric for Kurdyka-Łojasiewicz functions and general convergence rates
cites work: First-order methods almost always avoid strict saddle points
cites work: Nonconvex Robust Low-Rank Matrix Recovery
cites work: Behavior of accelerated gradient methods near critical points of nonconvex functions
cites work: Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
cites work: Nonconvergence to unstable points in urn models and stochastic approximations
cites work: Q3744918
cites work: Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
cites work: Complete Dictionary Recovery Over the Sphere I: Overview and the Geometric Picture
cites work: A geometric analysis of phase retrieval
cites work: Spurious Valleys in Two-layer Neural Network Optimization Landscapes
cites work: Gradient descent optimizes over-parameterized deep ReLU networks
DOI: 10.1007/S10957-024-02513-3
description: scientific article; zbMATH DE number 7960846