Neural collapse under cross-entropy loss

DOI10.1016/J.ACHA.2021.12.011MaRDI QIDQ2134115zbMATH OpenOpenAlexFDO

Authors Stefan Steinerberger, Jianfeng Lu

Publication date 6 May 2022

Published in Applied and Computational Harmonic Analysis (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2012.08465

zbMATH Keywords

neural networksframe potentialcross-entropy lossneural collapse

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) General harmonic expansions, frames (42C15)

Recommendations

• Neural collapse with unconstrained features
• Global minima of overparameterized neural networks
• Shaping the learning landscape in neural networks around wide flat minima
• The global optimization geometry of shallow linear neural networks
• Symmetry \& critical points for a model shallow neural network

Cites work

• Energy on spheres and discreteness of minimizing measures
• Finite normalized tight frames
• Optimal asymptotic bounds for spherical designs
• Positive definite functions on spheres
• Prevalence of neural collapse during the terminal phase of deep learning training
• The spectral function of an elliptic operator

Cited in

(2)

• Structure probing neural network deflation
• Neural Collapse with Cross-Entropy Loss

Describes a project that uses

Uses Software

• word2vec

This page was built for publication: Neural collapse under cross-entropy loss

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2134115)