Approximation spaces of deep neural networks

DOI10.1007/S00365-021-09543-4zbMath1491.82017arXiv1905.01208OpenAlexW2943191253MaRDI QIDQ2117336

Felix Voigtlaender, Gitta Kutyniok, Rémi Gribonval, Morten Nielsen

Publication date: 21 March 2022

Published in: Constructive Approximation (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1905.01208

zbMATH Keywords

Besov spacesapproximation spacespiecewise polynomialsdeep neural networksdirect estimatesinverse estimatessparsely connected networksReLU activation function

Mathematics Subject Classification ID

Nontrigonometric harmonic analysis involving wavelets and other special systems (42C40) Learning and adaptive systems in artificial intelligence (68T05) Neural nets applied to problems in time-dependent statistical mechanics (82C32) Abstract approximation theory (approximation in normed linear spaces and other abstract spaces) (41A65) Approximation by arbitrary nonlinear expressions; widths and entropy (41A46)

Related Items (25)

Neural network approximation ⋮ Learning with tree tensor networks: complexity estimates and model selection ⋮ Designing rotationally invariant neural networks from PDEs and variational methods ⋮ Training thinner and deeper neural networks: jumpstart regularization ⋮ Full error analysis for the training of deep neural networks ⋮ Simultaneous neural network approximation for smooth functions ⋮ Deep ReLU neural network approximation in Bochner spaces and applications to parametric PDEs ⋮ Deep learning methods for partial differential equations and related parameter identification problems ⋮ Approximation theory of tree tensor networks: tensorized univariate functions ⋮ Mesh-informed neural networks for operator learning in finite element spaces ⋮ Deep ReLU neural networks in high-dimensional approximation ⋮ Towards Lower Bounds on the Depth of ReLU Neural Networks ⋮ Universal regular conditional distributions via probabilistic transformers ⋮ Deep Learning in High Dimension: Neural Network Expression Rates for Analytic Functions in \(\pmb{L^2(\mathbb{R}^d,\gamma_d)}\) ⋮ Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class ⋮ A multivariate Riesz basis of ReLU neural networks ⋮ Collocation approximation by deep neural ReLU networks for parametric and stochastic PDEs with lognormal inputs ⋮ Revisiting Convolutional Neural Networks from the Viewpoint of Kernel-Based Methods ⋮ The universal approximation theorem for complex-valued neural networks ⋮ Learning ability of interpolating deep convolutional neural networks ⋮ Two-layer networks with the \(\text{ReLU}^k\) activation function: Barron spaces and derivative approximation ⋮ Approximation properties of residual neural networks for Kolmogorov PDEs ⋮ Sobolev-type embeddings for neural network approximation spaces ⋮ Applied harmonic analysis and data processing. Abstracts from the workshop held March 25--31, 2018 ⋮ A New Function Space from Barron Class and Application to Neural Network Approximation

Uses Software

• ImageNet
• VAMPnets
• U-Net
• AlexNet

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• Unnamed Item
• A mathematical introduction to compressive sensing
• Lower bounds for approximation by MLP neural networks
• Approximation and estimation bounds for artificial neural networks
• Multilayer feedforward networks are universal approximators
• Provable approximation properties for deep neural networks
• Approximation properties of a multilayered feedforward artificial neural network
• Degree of approximation by neural and translation networks with a single hidden layer
• Approximation of functions and their derivatives: A neural network implementation with applications
• Solving the Kolmogorov PDE by means of deep learning
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Nonparametric regression using deep neural networks with ReLU activation function
• Error bounds for approximations with deep ReLU networks
• Wavelet coorbit spaces viewed as decomposition spaces
• Affine synthesis onto \(L^p\) when \(0< p \leq 1\)
• Affine systems that span Lebesgue spaces
• Deep vs. shallow networks: An approximation theory perspective
• Topology with Applications
• Interpolation of Besov Spaces
• Besov Spaces on Domains in ℝ d
• Universal approximation bounds for superpositions of a sigmoidal function
• Neural Networks for Localized Approximation
• Generalized approximation spaces and applications
• Solving ill-posed inverse problems using iterative deep neural networks
• Maß- und Integrationstheorie
• Optimal Approximation with Sparsely Connected Deep Neural Networks
• Calculus With Applications
• Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography
• A logical calculus of the ideas immanent in nervous activity
• Approximation by superpositions of a sigmoidal function

This page was built for publication: Approximation spaces of deep neural networks