Provable approximation properties for deep neural networks
From MaRDI portal
Abstract: We discuss approximation of functions using deep neural nets. Given a function on a -dimensional manifold , we construct a sparsely-connected depth-4 neural network and bound its error in approximating . The size of the network depends on dimension and curvature of the manifold , the complexity of , in terms of its wavelet description, and only weakly on the ambient dimension . Essentially, our network computes wavelet functions, which are computed from Rectified Linear Units (ReLU)
Recommendations
- A note on the expressive power of deep rectified linear unit networks in high-dimensional spaces
- Error bounds for approximations with deep ReLU networks
- Deep network approximation characterized by number of neurons
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Approximation spaces of deep neural networks
Cites work
- scientific article; zbMATH DE number 108350 (Why is no real title available?)
- An introduction to manifolds
- Approximation and learning by greedy algorithms
- Approximation by superpositions of a sigmoidal function
- Comparison of worst case errors in linear and neural network approximation
- Harmonic analysis on spaces of homogeneous type. With a preface by Yves Meyer
- Networks and the best approximation property
- On the tractability of multivariate integration and approximation by neural networks
- Universal approximation bounds for superpositions of a sigmoidal function
Cited in
(98)- Provable Training of a ReLU Gate with an Iterative Non-Gradient Algorithm
- A New Function Space from Barron Class and Application to Neural Network Approximation
- Deep neural network expressivity for optimal stopping problems
- Approximation of functions from Korobov spaces by shallow neural networks
- Tractability of approximation by general shallow networks
- Side effects of learning from low-dimensional data embedded in a Euclidean space
- Any target function exists in a neighborhood of any sufficiently wide random network: a geometrical perspective
- On decision regions of narrow deep neural networks
- Error assessment of an adaptive finite elements -- neural networks method for an elliptic parametric PDE
- On PDE characterization of smooth hierarchical functions computed by neural networks
- Computing ground states of Bose-Einstein condensation by normalized deep neural network
- Approximation results for gradient flow trained shallow neural networks in \(1d\)
- Deep nonparametric estimation of intrinsic data structures by chart autoencoders: generalization error and robustness
- Approximation of Lipschitz Functions Using Deep Spline Neural Networks
- Wavelet neural networks functional approximation and application
- Neural network approximation and estimation of classifiers with classification boundary in a Barron class
- Surgery of frames in Hilbert spaces
- Neural network approximation of continuous functions in high dimensions with applications to inverse problems
- Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks
- A note on the applications of one primary function in deep neural networks
- Deep neural networks can stably solve high-dimensional, noisy, non-linear inverse problems
- Quadratic Neural Networks for Solving Inverse Problems
- Designing universal causal deep learning models: The geometric (Hyper)transformer
- Approximation with one-bit polynomials in Bernstein form
- Approximation properties of ridge functions and extreme learning machines
- Approximation of smoothness classes by deep rectifier networks
- Limitations of neural network training due to numerical instability of backpropagation
- Approximation of smooth functionals using deep ReLU networks
- Optimal approximation with sparsely connected deep neural networks
- Gabor neural networks with proven approximation properties
- On the proof complexity of deep inference
- Theory of deep convolutional neural networks. II: Spherical analysis
- Breaking the curse of dimensionality with convex neural networks
- High-dimensional distribution generation through deep neural networks
- Theory of deep convolutional neural networks. III: Approximating radial functions
- Neural network approximation
- Approximation spaces of deep neural networks
- Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations
- A note on the expressive power of deep rectified linear unit networks in high-dimensional spaces
- Theory of deep convolutional neural networks: downsampling
- Butterfly-net: optimal function representation based on convolutional neural networks
- Multi-scale deep neural network (MscaleDNN) for solving Poisson-Boltzmann equation in complex domains
- Depth separations in neural networks: what is actually being separated?
- Approximations with deep neural networks in Sobolev time-space
- Approximation of classifiers by deep perceptron networks
- A direct approach for function approximation on data defined manifolds
- Topological properties of the set of functions generated by neural networks of fixed size
- Construct Deep Neural Networks based on Direct Sampling Methods for Solving Electrical Impedance Tomography
- Approximation Analysis of Convolutional Neural Networks
- A deep network construction that adapts to intrinsic dimensionality beyond the domain
- scientific article; zbMATH DE number 7626714 (Why is no real title available?)
- Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class
- Deep learning volatility: a deep neural network perspective on pricing and calibration in (rough) volatility models
- Universal approximation with quadratic deep networks
- Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs
- Approximation in shift-invariant spaces with deep ReLU neural networks
- Representation formulas and pointwise properties for Barron functions
- Deep ReLU neural networks in high-dimensional approximation
- Deep ReLU Networks Overcome the Curse of Dimensionality for Generalized Bandlimited Functions
- Universal approximation of symmetric and anti-symmetric functions
- Machine learning based data retrieval for inverse scattering problems with incomplete data
- Overall error analysis for the training of deep neural networks via stochastic gradient descent with random initialisation
- A Proof that Artificial Neural Networks Overcome the Curse of Dimensionality in the Numerical Approximation of Black–Scholes Partial Differential Equations
- Deep Neural Network Approximation Theory
- A theoretical analysis of deep neural networks and parametric PDEs
- Analysis of time-frequency scattering transforms
- Approximation properties of deep ReLU CNNs
- MgNet: a unified framework of multigrid and convolutional neural network
- Banach space representer theorems for neural networks and ridge splines
- Deep hedging
- Rapid estimation of permeability from digital rock using 3D convolutional neural network
- Holomorphic feedforward networks
- Deep neural networks for rotation-invariance approximation and learning
- Growing axons: greedy learning of neural networks with application to function approximation
- Approximation of nonlinear functionals using deep ReLU networks
- Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
- A measure theoretical approach to the mean-field maximum principle for training NeurODEs
- Stable recovery of entangled weights: towards robust identification of deep neural networks from minimal samples
- Nonlinear approximation and (deep) ReLU networks
- Equivalence of approximation by convolutional neural networks and fully-connected networks
- Numerical solution of the parametric diffusion equation by deep neural networks
- Error bounds for approximations with deep ReLU networks
- Analysis of the generalization error: empirical risk minimization over deep artificial neural networks overcomes the curse of dimensionality in the numerical approximation of Black-Scholes partial differential equations
- Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
- On the approximation of rough functions with deep neural networks
- PDE-Net 2.0: learning PDEs from data with a numeric-symbolic hybrid deep network
- The gap between theory and practice in function approximation with deep neural networks
- Universality of deep convolutional neural networks
- Deep network approximation characterized by number of neurons
- Robust and resource-efficient identification of two hidden layer neural networks
- Product formalisms for measures on spaces with binary tree structures: representation, visualization, and multiscale noise
- Deep learning methods for partial differential equations and related parameter identification problems
- On the capacity of deep generative networks for approximating distributions
- Full error analysis for the training of deep neural networks
- Deep ReLU networks and high-order finite element methods
- Rates of approximation by ReLU shallow neural networks
- The construction and approximation of ReLU neural network operators
- Approximation rates for neural networks with encodable weights in smoothness spaces
This page was built for publication: Provable approximation properties for deep neural networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1742817)
