Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks

DOI10.1214/23-AAP2010MaRDI QIDQ6590448FDOQ6590448

Authors: Zhichao Wang, Yizhe Zhu 

Publication date: 21 August 2024

Published in: The Annals of Applied Probability (Search for Journal in Brave)

Recommendations

• Nonlinear random matrix theory for deep learning
• A random matrix approach to neural networks
• Eigenvalue distribution of some nonlinear models of random matrices
• Infinite-width limit of deep linear neural networks
• Random neural networks in the infinite width limit as Gaussian processes

zbMATH Keywords

random matrix theoryneural networksneural tangent kernelrandom feature regression

Mathematics Subject Classification ID

Ridge regression; shrinkage estimators (Lasso) (62J07) Artificial neural networks and deep learning (68T07) Random matrices (probabilistic aspects) (60B20)

Cites Work

• Spectral analysis of large dimensional random matrices
• Lectures on the Combinatorics of Free Probability
• High-dimensional probability. An introduction with applications in data science
• Concentration inequalities. A nonasymptotic theory of independence
• An Inverse Matrix Adjustment Arising in Discriminant Analysis
• Hanson-Wright inequality and sub-Gaussian concentration
• On the equivalence between kernel quadrature rules and random feature expansions
• User-friendly tail bounds for sums of random matrices
• Title not available (Why is that?)
• A Bound on Tail Probabilities for Quadratic Forms in Independent Random Variables
• A note on the Hanson-Wright inequality for random vectors with dependencies
• Convergence to the semicircle law
• Testing the sphericity of a covariance matrix when the dimension is much larger than the sample size
• The smallest eigenvalue of a large dimensional Wishart matrix
• Partial transposition of random states and non-centered semicircular distributions
• The limiting distributions of eigenvalues of sample correlation matrices
• Strong convergence of ESD for the generalized sample covariance matrices when \(p/n \rightarrow 0\)
• The limiting spectral distribution of the product of the Wigner matrix and a nonnegative definite matrix
• Convergence of the largest eigenvalue of normalized sample covariance matrices when \(p\) and \(n\) both tend to infinity with their ratio converging to zero
• Alice and Bob Meet Banach
• Just interpolate: kernel ``ridgeless regression can generalize
• A random matrix approach to neural networks
• Limiting spectral distribution of normalized sample covariance matrices with \(p/n\to 0\)
• CLT for linear spectral statistics of normalized sample covariance matrices with the dimension much larger than the sample size
• Limiting spectral distribution of renormalized separable sample covariance matrices when \(p/n\to 0\)
• Eigenvalue distribution of some nonlinear models of random matrices
• A note on the Pennington-Worah distribution
• The Generalization Error of Random Features Regression: Precise Asymptotics and the Double Descent Curve
• The interpolation phase transition in neural networks: memorization and generalization under lazy training
• Generalization error of random feature and kernel methods: hypercontractivity and kernel matrix concentration
• The PPT square conjecture holds generically for some classes of independent states
• Deep learning: a statistical viewpoint
• Title not available (Why is that?)
• Learning curves of generic features maps for realistic datasets with a teacher-student model*
• Asymptotic freeness of layerwise Jacobians caused by invariance of multilayer perceptron: the Haar orthogonal case
• Asymptotic normality for eigenvalue statistics of a general sample covariance matrix when \(p/n \to \infty\) and applications
• Large-dimensional random matrix theory and its applications in deep learning and wireless communications
• Spiked singular values and vectors under extreme aspect ratios
• Universality Laws for High-Dimensional Learning With Random Features

This page was built for publication: Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6590448)