Infinite-width limit of deep linear neural networks
From MaRDI portal
Publication:6587580
DOI10.1002/CPA.22200MaRDI QIDQ6587580FDOQ6587580
Authors: Lénaïc Chizat, Maria Colombo, Xavier Fernández-Real, Alessio Figalli 
Publication date: 14 August 2024
Published in: Communications on Pure and Applied Mathematics (Search for Journal in Brave)
Recommendations
- Wide neural networks of any depth evolve as linear models under gradient descent *
- Random neural networks in the infinite width limit as Gaussian processes
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- Mean Field Analysis of Deep Neural Networks
- Mean-field limits of trained weights in deep learning: a dynamical systems perspective
Artificial neural networks and deep learning (68T07) Random matrices (probabilistic aspects) (60B20)
Cites Work
- High-dimensional probability. An introduction with applications in data science
- Representations for partially exchangeable arrays of random variables
- Bayesian learning for neural networks
- The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
- An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
- Approximation and estimation bounds for artificial neural networks
- A mean field view of the landscape of two-layer neural networks
- A mathematical theory of semantic development in deep neural networks
- Mean field analysis of neural networks: a law of large numbers
- Products of many large random matrices and gradients in deep neural networks
- Disentangling feature and lazy training in deep neural networks
- Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
- The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows
- Deep Linear Networks for Matrix Completion—an Infinite Depth Limit
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- Gradient flows on graphons: existence, convergence, continuity equations
Cited In (10)
- Mean-field limits of trained weights in deep learning: a dynamical systems perspective
- Deep stable neural networks: large-width asymptotics and convergence rates
- Random neural networks in the infinite width limit as Gaussian processes
- Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks
- Exact learning dynamics of deep linear networks with prior knowledge
- Self-consistent dynamical field theory of kernel evolution in wide neural networks
- -Stable convergence of heavy-/light-tailed infinitely wide neural networks
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- Deep Linear Networks for Matrix Completion—an Infinite Depth Limit
- Homotopy relaxation training algorithms for infinite-width two-layer ReLU neural networks
This page was built for publication: Infinite-width limit of deep linear neural networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6587580)
