Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry*

DOI10.1088/1742-5468/AC9832OpenAlexW4309880056MaRDI QIDQ5055419FDOQ5055419

Authors: Fabrizio Pittorino, Antonio Ferraro, Gabriele Perugini, Christoph Feinauer, Carlo Baldassi, Riccardo Zecchina 

Publication date: 13 December 2022

Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2202.03038

zbMATH Keywords

deep learningmachine learning

Mathematics Subject Classification ID

Statistical mechanics, structure of matter (82-XX)

Cites Work

• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• Entropy-SGD: biasing gradient descent into wide valleys
• Flat Minima
• Shaping the learning landscape in neural networks around wide flat minima
• The inverse variance–flatness relation in stochastic gradient descent is critical for finding flat minima
• Entropic gradient descent algorithms and wide flat minima*

Uses Software

• SciPy

This page was built for publication: Deep networks on toroids: removing symmetries reveals the structure of flat regions in the landscape geometry*

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5055419)