Infinite-width limit of deep linear neural networks
From MaRDI portal
Publication:6587580
Recommendations
- Wide neural networks of any depth evolve as linear models under gradient descent *
- Random neural networks in the infinite width limit as Gaussian processes
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- Mean Field Analysis of Deep Neural Networks
- Mean-field limits of trained weights in deep learning: a dynamical systems perspective
Cites work
- A mathematical theory of semantic development in deep neural networks
- A mean field view of the landscape of two-layer neural networks
- An iterative construction of solutions of the TAP equations for the Sherrington-Kirkpatrick model
- Approximation and estimation bounds for artificial neural networks
- Bayesian learning for neural networks
- Deep Linear Networks for Matrix Completion—an Infinite Depth Limit
- Disentangling feature and lazy training in deep neural networks
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- Gradient flows on graphons: existence, convergence, continuity equations
- High-dimensional probability. An introduction with applications in data science
- Learning deep linear neural networks: Riemannian gradient flows and convergence to global minimizers
- Mean field analysis of neural networks: a law of large numbers
- Products of many large random matrices and gradients in deep neural networks
- Representations for partially exchangeable arrays of random variables
- The Continuous Formulation of Shallow Neural Networks as Wasserstein-Type Gradient Flows
- The Dynamics of Message Passing on Dense Graphs, with Applications to Compressed Sensing
Cited in
(10)- Mean-field limits of trained weights in deep learning: a dynamical systems perspective
- Deep stable neural networks: large-width asymptotics and convergence rates
- Random neural networks in the infinite width limit as Gaussian processes
- Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks
- Exact learning dynamics of deep linear networks with prior knowledge
- Self-consistent dynamical field theory of kernel evolution in wide neural networks
- -Stable convergence of heavy-/light-tailed infinitely wide neural networks
- Gradient descent on infinitely wide neural networks: global convergence and generalization
- Deep Linear Networks for Matrix Completion—an Infinite Depth Limit
- Homotopy relaxation training algorithms for infinite-width two-layer ReLU neural networks
This page was built for publication: Infinite-width limit of deep linear neural networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6587580)
