Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions

DOI10.1007/s00332-022-09823-8zbMath1491.68179arXiv2103.10922OpenAlexW4284712609MaRDI QIDQ2156337

Patrick Cheridito, Florian Rossmannek, Arnulf Jentzen

Publication date: 18 July 2022

Published in: Journal of Nonlinear Science (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2103.10922

zbMATH Keywords

neural networksnonconvex optimizationlandscape analysisshallow networksloss surface

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26)

Related Items (2)

A proof of convergence for stochastic gradient descent in the training of artificial neural networks with ReLU activation for constant target functions ⋮ Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation

Cites Work

• Unnamed Item
• Topological properties of the set of functions generated by neural networks of fixed size
• A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions
• First-order methods almost always avoid strict saddle points
• Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
• Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions
• Spurious Valleys in Two-layer Neural Network Optimization Landscapes

This page was built for publication: Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions