Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces

DOI10.1007/S10208-023-09607-WMaRDI QIDQ6592113zbMATH OpenFDO

Authors Philipp Grohs, Felix Voigtlaender

Publication date 24 August 2024

Published in Foundations of Computational Mathematics (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://doi.org/10.1007/S10208-023-09607-W

zbMATH Keywords

approximation spacesdeep neural networksGelfand numbersinformation based complexityrandomized approximationtheory-to-computational gaps

Mathematics Subject Classification ID

Approximation by arbitrary nonlinear expressions; widths and entropy (41A46) Abstract approximation theory (approximation in normed linear spaces and other abstract spaces) (41A65) Artificial neural networks and deep learning (68T07)

Recommendations

• Sampling complexity of deep approximation spaces
• The gap between theory and practice in function approximation with deep neural networks
• Hardness results for neural network approximation problems
• Error bounds for approximations with deep ReLU networks
• Expressivity of Deep Neural Networks

Cites work

• A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
• Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
• Approximation spaces of deep neural networks
• Deep Neural Network Approximation Theory
• Foundations of machine learning
• High-dimensional probability. An introduction with applications in data science
• scientific article; zbMATH DE number 50401 (Why is no real title available?)
• scientific article; zbMATH DE number 1324223 (Why is no real title available?)
• scientific article; zbMATH DE number 477682 (Why is no real title available?)
• scientific article; zbMATH DE number 515830 (Why is no real title available?)
• scientific article; zbMATH DE number 1076783 (Why is no real title available?)
• scientific article; zbMATH DE number 7064043 (Why is no real title available?)
• Infinite dimensional analysis. A hitchhiker's guide.
• Neural network approximation
• On the mathematical foundations of learning
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Optimal approximation with sparsely connected deep neural networks
• Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
• Solving inverse problems using data-driven models
• The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
• The gap between theory and practice in function approximation with deep neural networks
• Understanding machine learning. From theory to algorithms

Cited in

(8)

• PAC-learning gains of Turing machines over circuits and neural networks
• Sampling complexity of deep approximation spaces
• Sample complexity bounds for the local convergence of least squares approximation
• Computational multiscale methods. Abstracts from the workshop held April 27 -- May 2, 2025
• Approximation results for gradient flow trained neural networks
• Compositional sparsity, approximation classes, and parametric transport equations
• Time-frequency analysis for neural networks
• Stable learning using spiking neural networks equipped with affine encoders and decoders

This page was built for publication: Proof of the theory-to-practice gap in deep learning via sampling complexity bounds for neural network approximation spaces

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6592113)