Landscape and training regimes in deep learning
From MaRDI portal
Publication:2231925
DOI10.1016/J.PHYSREP.2021.04.001OpenAlexW3153303803MaRDI QIDQ2231925FDOQ2231925
Mario Geiger, Matthieu Wyart, Leonardo Petrini
Publication date: 30 September 2021
Published in: Physics Reports (Search for Journal in Brave)
Full work available at URL: https://doi.org/10.1016/j.physrep.2021.04.001
Recommendations
- Archetypal landscapes for deep neural networks
- Optimization Landscape of Neural Networks
- Shaping the learning landscape in neural networks around wide flat minima
- Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity
- Deep learning
- Deep learning
- Learning with deep cascades
- Exploring strategies for training deep neural networks
- Disentangling feature and lazy training in deep neural networks
deep learningcurse of dimensionalityneural networksfeature learningjammingneural tangent kernellazy trainingloss landscape
Cites Work
- Bayesian learning for neural networks
- Exact theory of dense amorphous hard spheres in high dimension. III. The full replica symmetry breaking solution
- Title not available (Why is that?)
- Mean field analysis of neural networks: a central limit theorem
- A mean field view of the landscape of two-layer neural networks
- Reconciling modern machine-learning practice and the classical bias–variance trade-off
- Breaking the Curse of Dimensionality with Convex Neural Networks
- High-dimensional dynamics of generalization error in neural networks
- Linearized two-layers neural networks in high dimension
- Comparing dynamics: deep neural networks versus glassy systems
- Mean Field Analysis of Neural Networks: A Law of Large Numbers
- Disentangling feature and lazy training in deep neural networks
- Wide neural networks of any depth evolve as linear models under gradient descent *
- On the information bottleneck theory of deep learning
- Scaling description of generalization with number of parameters in deep learning
- A jamming transition from under- to over-parametrization affects generalization in deep learning
- Global Minima of Overparameterized Neural Networks
- The simplest model of jamming
- Universality of jamming of nonspherical particles
- Asymptotic learning curves of kernel methods: empirical data versus teacher–student paradigm
- Surfing on minima of isostatic landscapes: avalanches and unjamming transition
Cited In (5)
- Dynamical mean field theory for models of confluent tissues and beyond
- Learning sparse features can lead to overfitting in neural networks
- Relative stability toward diffeomorphisms indicates performance in deep nets*
- DANTE: deep alternations for training neural networks
- Revisiting Landscape Analysis in Deep Neural Networks: Eliminating Decreasing Paths to Infinity
Uses Software
This page was built for publication: Landscape and training regimes in deep learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2231925)
