Traces of class/cross-class structure pervade deep learning spectra

MaRDI QIDQ5149053zbMATH OpenFDO

Authors Vardan Papyan

Publication date 5 February 2021

Full work available at URL https://arxiv.org/abs/2008.11865, https://jmlr.csail.mit.edu/papers/v21/20-933.html

zbMATH Keywords

deep learningHessianlow-rank approximationmultinomial logistic regressionspectral analysis

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Recommendations

• Pathological Spectra of the Fisher Information Metric and Its Variants in Deep Neural Networks
• The Modern Mathematics of Deep Learning
• Universal characteristics of deep neural network loss surfaces from random matrix theory
• Spectral plot properties: towards a qualitative classification of networks
• Convergence rates of deep ReLU networks for multiclass classification

Cited in

(5)

• Spectral alignment of stochastic gradient descent for high-dimensional classification tasks
• Unraveling the Hessian: a key to smooth convergence in loss function landscapes
• A geometric modeling of Occam's razor in deep learning
• Stochastic gradient descent in high dimensions for multi-spiked tensor PCA
• Local geometry of high-dimensional mixture models: effective spectral theory and dynamical transitions

This page was built for publication: Traces of class/cross-class structure pervade deep learning spectra

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5149053)