Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry*

DOI10.1088/1742-5468/AC9832OpenAlexW4309880056MaRDI QIDQ5055419FDOQ5055419

Authors: Fabrizio Pittorino, Antonio Ferraro, Gabriele Perugini, Christoph Feinauer, Carlo Baldassi, Riccardo Zecchina 

Publication date: 13 December 2022

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2202.03038

Recommendations

• Shaping the learning landscape in neural networks around wide flat minima
• Entropic gradient descent algorithms and wide flat minima*
• Clustering of solutions in the symmetric binary perceptron
• Archetypal landscapes for deep neural networks
• Wide flat minima and optimal generalization in classifying high-dimensional Gaussian mixtures

zbMATH Keywords

deep learningmachine learning

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Cites Work

• Reconciling modern machine-learning practice and the classical bias-variance trade-off
• Entropy-SGD: biasing gradient descent into wide valleys
• Flat Minima
• Shaping the learning landscape in neural networks around wide flat minima
• The inverse variance-flatness relation in stochastic gradient descent is critical for finding flat minima
• Entropic gradient descent algorithms and wide flat minima*

Uses Software

• SciPy

This page was built for publication: Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry*

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5055419)