Neural collapse under cross-entropy loss

DOI10.1016/J.ACHA.2021.12.011OpenAlexW4205698766MaRDI QIDQ2134115FDOQ2134115

Authors: Stefan Steinerberger, Jianfeng Lu 

Publication date: 6 May 2022

Published in: Applied and Computational Harmonic Analysis (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2012.08465

Recommendations

• Neural collapse with unconstrained features
• Global minima of overparameterized neural networks
• Shaping the learning landscape in neural networks around wide flat minima
• The global optimization geometry of shallow linear neural networks
• Symmetry \& critical points for a model shallow neural network

zbMATH Keywords

neural networksframe potentialcross-entropy lossneural collapse

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) General harmonic expansions, frames (42C15)

Cites Work

• The spectral function of an elliptic operator
• Finite normalized tight frames
• Positive definite functions on spheres
• Optimal asymptotic bounds for spherical designs
• Energy on spheres and discreteness of minimizing measures
• Prevalence of neural collapse during the terminal phase of deep learning training

Cited In (2)

• Neural Collapse with Cross-Entropy Loss
• Structure probing neural network deflation

Uses Software

• word2vec

This page was built for publication: Neural collapse under cross-entropy loss

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2134115)