Mean-field limits of trained weights in deep learning: a dynamical systems perspective

DOI10.14658/PUPJ-DRNA-2022-3-12MaRDI QIDQ6556653zbMATH OpenFDO

Authors Alexandre Smirnov, B. Hamzi, Houman Owhadi

Publication date 17 June 2024

Published in Dolomites Research Notes on Approximation (Search for Journal in Brave)

Full work available at URL https://doi.org/10.14658/PUPJ-DRNA-2022-3-12

Mathematics Subject Classification ID

Approximation methods and numerical treatment of dynamical systems (37M99) Approximation by other special function classes (41A30) Artificial neural networks and deep learning (68T07)

Recommendations

• Continuous limits of residual neural networks in case of large input data
• Deep limits of residual neural networks
• Mean Field Analysis of Deep Neural Networks
• Selection dynamics for deep neural networks
• Infinite-width limit of deep linear neural networks

Cites work

• Deep learning
• Discrete mechanics and variational integrators
• Hedonic housing prices and the demand for clean air
• scientific article; zbMATH DE number 4128577 (Why is no real title available?)
• Kernel methods in machine learning
• Multisymplectic geometry, variational integrators, and nonlinear PDEs
• Operator-valued kernels for learning from functional response data
• Support Vector Machines

Cited in

(10)

• Deep limits of residual neural networks
• Deep learning in random neural fields: numerical experiments via neural tangent kernel
• Deep learning via dynamical systems: an approximation perspective
• 5-th Dolomites Workshop on Constructive Approximation and Applications -- special issue dedicated to Robert Schaback on the occasion of his 75th birthday
• Infinite-width limit of deep linear neural networks
• Self-consistent dynamical field theory of kernel evolution in wide neural networks
• Gaussian processes simplify differential equations
• Analysis of structured deep kernel networks
• Operator theory, kernels, and feedforward neural networks
• Kernel sum of squares for data adapted kernel learning of dynamical systems from data: a global optimization approach

This page was built for publication: Mean-field limits of trained weights in deep learning: a dynamical systems perspective

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6556653)