Random neural networks in the infinite width limit as Gaussian processes

DOI10.1214/23-aap1933arXiv2107.01562MaRDI QIDQ6138923

Boris Hanin

Publication date: 16 January 2024

Published in: The Annals of Applied Probability (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/2107.01562

zbMATH Keywords

neural networksGaussian processeslimit theorems

Mathematics Subject Classification ID

Gaussian processes (60G15) Artificial neural networks and deep learning (68T07)

Cites Work

• On the distribution of the roots of certain symmetric matrices
• Addition of certain non-commuting random variables
• Ergodic theory of differentiable dynamical systems
• Estimation of moments of sums of independent real random variables
• Bayesian learning for neural networks
• Non-asymptotic results for singular values of Gaussian matrix products
• Nonlinear approximation and (deep) ReLU networks
• Surprises in high-dimensional ridgeless least squares interpolation
• Fluctuations of \(\beta\)-Jacobi product processes
• Products of many large random matrices and gradients in deep neural networks
• A note on the Pennington-Worah distribution
• Universal microscopic correlation functions for products of independent Ginibre matrices
• Lectures on the Combinatorics of Free Probability
• Gaussian fluctuations for products of random matrices
• The Principles of Deep Learning Theory
• Benign overfitting in linear regression
• Universal microscopic correlation functions for products of truncated unitary matrices
• Reconciling modern machine-learning practice and the classical bias–variance trade-off
• Products of Random Matrices
• Noncommuting Random Products
• Neural network approximation