Error bounds for approximations with deep ReLU networks
From MaRDI portal
Publication:2292227
Abstract: We study expressive power of shallow and deep neural networks with piece-wise linear activation functions. We establish new rigorous upper and lower bounds for the network complexity in the setting of approximations in Sobolev spaces. In particular, we prove that deep ReLU networks more efficiently approximate smooth functions than shallow networks. In the case of approximations of 1D Lipschitz functions we describe adaptive depth-6 network architectures more efficient than the standard shallow architecture.
Recommendations
- Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
- Nonlinear approximation and (deep) ReLU networks
- Provable approximation properties for deep neural networks
- New error bounds for deep ReLU networks using sparse grids
- Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
Cites work
- Approximation properties of a multilayered feedforward artificial neural network
- Bounding the Vapnik-Chervonenkis dimension of concept classes parameterized by real numbers
- scientific article; zbMATH DE number 5602305 (Why is no real title available?)
- scientific article; zbMATH DE number 1776257 (Why is no real title available?)
- scientific article; zbMATH DE number 1405266 (Why is no real title available?)
- Lower Bounds for Approximation by Nonlinear Manifolds
- On the expressive power of deep architectures
- On the near optimality of the stochastic approximation of smooth functions by neural networks
- On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities
- Optimal nonlinear approximation
- The Solovay--Kitaev algorithm
Cited in
(only showing first 100 items - show all)- Provable approximation properties for deep neural networks
- Approximation rates for neural networks with general activation functions
- Exponential convergence of the deep neural network approximation for analytic functions
- A neural network-based framework for financial model calibration
- Rademacher complexity and the generalization error of residual networks
- Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs
- Topological properties of the set of functions generated by neural networks of fixed size
- Estimation error analysis of deep learning on the regression problem on the variable exponent Besov space
- Recurrent neural networks as optimal mesh refinement strategies
- Numerical solution of the parametric diffusion equation by deep neural networks
- Model reduction and neural networks for parametric PDEs
- Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
- Fast generalization error bound of deep learning without scale invariance of activation functions
- Approximation rates for neural networks with encodable weights in smoothness spaces
- Theory of deep convolutional neural networks. II: Spherical analysis
- Optimal approximation rate of ReLU networks in terms of width and depth
- Constructive deep ReLU neural network approximation
- Infinite-dimensional inverse problems with finite measurements
- Rates of approximation by neural network interpolation operators
- Function approximation by deep neural networks with parameters \(\{0, \pm \frac{1}{2}, \pm 1,2\}\)
- Sparsest piecewise-linear regression of one-dimensional data
- The construction and approximation of ReLU neural network operators
- On the approximation of rough functions with deep neural networks
- Compact mixed-integer programming formulations in quadratic optimization
- CAS4DL: Christoffel adaptive sampling for function approximation via deep learning
- Solving parametric partial differential equations with deep rectified quadratic unit neural networks
- Approximation of functions from korobov spaces by deep convolutional neural networks
- Error bounds for ReLU networks with depth and width parameters
- Deep learning architectures for nonlinear operator functions and nonlinear inverse problems
- DNN expression rate analysis of high-dimensional PDEs: application to option pricing
- A theoretical analysis of deep neural networks and parametric PDEs
- Nonlinear approximation and (deep) ReLU networks
- Depth separations in neural networks: what is actually being separated?
- Approximation spaces of deep neural networks
- Exponential ReLU DNN expression of holomorphic maps in high dimension
- Best \(k\)-layer neural network approximations
- High-order approximation rates for shallow neural networks with cosine and \(\mathrm{ReLU}^k\) activation functions
- Adaptive two-layer ReLU neural network. I: Best least-squares approximation
- Structure probing neural network deflation
- Deep least-squares methods: an unsupervised learning-based numerical method for solving elliptic PDEs
- Int-Deep: a deep learning initialized iterative method for nonlinear problems
- SelectNet: self-paced learning for high-dimensional partial differential equations
- Thermodynamically consistent physics-informed neural networks for hyperbolic systems
- Convergence rates of deep ReLU networks for multiclass classification
- Neural network training using \(\ell_1\)-regularization and bi-fidelity data
- On sharpness of an error bound for deep ReLU network approximation
- Deep solution operators for variational inequalities via proximal neural networks
- Deep learning for the partially linear Cox model
- Approximation properties of deep ReLU CNNs
- ReLU deep neural networks from the hierarchical basis perspective
- Scientific machine learning through physics-informed neural networks: where we are and what's next
- A generative model for fBm with deep ReLU neural networks
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Gradient descent optimizes over-parameterized deep ReLU networks
- Nonlinear approximation via compositions
- An analysis of training and generalization errors in shallow and deep networks
- On the minimax optimality and superiority of deep neural network learning over sparse parameter spaces
- Theory of deep convolutional neural networks: downsampling
- Function approximation by deep networks
- Nonparametric regression using deep neural networks with ReLU activation function
- A functional equation with polynomial solutions and application to neural networks
- A mesh-free method for interface problems using the deep learning approach
- Optimal nonparametric inference via deep neural network
- A global universality of two-layer neural networks with ReLU activations
- Limitations of shallow nets approximation
- Universality of deep convolutional neural networks
- A machine learning framework for data driven acceleration of computations of differential equations
- Neural network with unbounded activation functions is universal approximator
- Learning the mapping \(\mathbf{x}\mapsto \sum\limits_{i=1}^d x_i^2\): the cost of finding the needle in a haystack
- Optimal stable nonlinear approximation
- Wasserstein generative adversarial uncertainty quantification in physics-informed neural networks
- Sparse approximation of triangular transports. I: The finite-dimensional case
- Sparse approximation of triangular transports. II: The infinite-dimensional case
- DeepParticle: learning invariant measure by a deep neural network minimizing Wasserstein distance on data generated from an interacting particle method
- Quantitative estimates for neural network operators implied by the asymptotic behaviour of the sigmoidal activation functions
- Estimation of a regression function on a manifold by fully connected deep neural networks
- Neural control of discrete weak formulations: Galerkin, least squares \& minimal-residual methods with quasi-optimal weights
- A deep first-order system least squares method for solving elliptic PDEs
- Space-time error estimates for deep neural network approximations for differential equations
- A convenient infinite dimensional framework for generative adversarial learning
- Side effects of learning from low-dimensional data embedded in a Euclidean space
- The universal approximation theorem for complex-valued neural networks
- Growing axons: greedy learning of neural networks with application to function approximation
- Deep learning for inverse problems with unknown operator
- Computation and learning in high dimensions. Abstracts from the workshop held August 1--7, 2021 (hybrid meeting)
- Learning elliptic partial differential equations with randomized linear algebra
- Greedy training algorithms for neural networks and applications to PDEs
- Approximation with one-bit polynomials in Bernstein form
- Deep vs. shallow networks: an approximation theory perspective
- On Geometric Invariants, Learning, and Recognition of Shapes and Forms
- ReLU networks are universal approximators via piecewise linear or constant functions
- Approximation by Combinations of ReLU and Squared ReLU Ridge Functions With <inline-formula> <tex-math notation="LaTeX">$\ell^1$ </tex-math> </inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\ell^0$ </tex-math> </inline-formula> Controls
- Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
- Transport analysis of infinitely deep neural network
- Adaptive low-rank approximations for operator equations: accuracy control and computational complexity
- The gap between theory and practice in function approximation with deep neural networks
- Deep network with approximation error being reciprocal of width to power of square root of depth
- scientific article; zbMATH DE number 7387621 (Why is no real title available?)
- scientific article; zbMATH DE number 7387622 (Why is no real title available?)
- A multi-level procedure for enhancing accuracy of machine learning algorithms
This page was built for publication: Error bounds for approximations with deep ReLU networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2292227)
