Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry*

DOI10.1088/1742-5468/AC9832MaRDI QIDQ5055419zbMATH OpenOpenAlexFDO

Authors Fabrizio Pittorino, Antonio Ferraro, Gabriele Perugini, Christoph Feinauer, Carlo Baldassi, Riccardo Zecchina

Publication date 13 December 2022

Published in Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2202.03038

zbMATH Keywords

deep learningmachine learning

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Recommendations

• Shaping the learning landscape in neural networks around wide flat minima
• Entropic gradient descent algorithms and wide flat minima*
• Clustering of solutions in the symmetric binary perceptron
• Archetypal landscapes for deep neural networks
• Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures

Cites work

• Entropic gradient descent algorithms and wide flat minima*
• Entropy-SGD: biasing gradient descent into wide valleys
• Flat Minima
• Reconciling modern machine-learning practice and the classical bias-variance trade-off
• Shaping the learning landscape in neural networks around wide flat minima
• The inverse variance-flatness relation in stochastic gradient descent is critical for finding flat minima

Cited in

(1)

• Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures

Describes a project that uses

Uses Software

• SciPy

This page was built for publication: Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry*

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5055419)