Deep ReLU neural networks in high-dimensional approximation
From MaRDI portal
Abstract: We study the computation complexity of deep ReLU (Rectified Linear Unit) neural networks for the approximation of functions from the H"older-Zygmund space of mixed smoothness defined on the -dimensional unit cube when the dimension may be very large. The approximation error is measured in the norm of isotropic Sobolev space. For every function from the H"older-Zygmund space of mixed smoothness, we explicitly construct a deep ReLU neural network having an output that approximates with a prescribed accuracy , and prove tight dimension-dependent upper and lower bounds of the computation complexity of this approximation, characterized as the size and the depth of this deep ReLU neural network, explicitly in and . The proof of these results are in particular, relied on the approximation by sparse-grid sampling recovery based on the Faber series.
The paper devotes to the approximation of functions from the Hölder-Zygmund space of mixed smoothness defined on the unit cube of a high-dimensional Euclidean space with deep ReLU (Rectified Linear Unit) neural networks. The focus of the study is on computation complexity the network in order to attain a prescribed accuracy. The computational complexity is measured by the size and depth of the deep ReLU neural network, where the depth and size are defined to be the number of layers and the number of nonzero weights of the network, respectively. The main contribution of the paper is to give dimension-dependent upper and lower bounds of the computation complexity of a deep ReLU neural networks that approximates a given function from the Hölder-Zygmund space of mixed smoothness with a prescribed accuracy.
Recommendations
- A note on the expressive power of deep rectified linear unit networks in high-dimensional spaces
- Error bounds for approximations with deep ReLU networks
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Deep network approximation characterized by number of neurons
- Provable approximation properties for deep neural networks
Cites work
- \(N\)-widths and \(\varepsilon \)-dimensions for high-dimensional approximations
- A logical calculus of the ideas immanent in nervous activity
- A note on the complexity of solving Poisson's equation for spaces of bounded mixed derivatives
- Approximation of mixed order Sobolev functions on the \(d\)-torus: asymptotics, preasymptotics, and \(d\)-dependence
- Approximation spaces of deep neural networks
- B-spline quasi-interpolant representations and sampling recovery of functions with mixed smoothness
- Bases in function spaces, sampling, discrepancy, numerical integration
- Data mining with sparse grids
- Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
- Deep ReLU networks and high-order finite element methods
- Dimension-dependent error estimates for sampling recovery on Smolyak grids based on B-spline quasi-interpolation
- Error bounds for approximations with deep ReLU networks
- Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
- Exponential convergence of the deep neural network approximation for analytic functions
- Function spaces with dominating mixed smoothness
- scientific article; zbMATH DE number 3126094 (Why is no real title available?)
- scientific article; zbMATH DE number 4080153 (Why is no real title available?)
- scientific article; zbMATH DE number 3481257 (Why is no real title available?)
- scientific article; zbMATH DE number 3596197 (Why is no real title available?)
- scientific article; zbMATH DE number 477682 (Why is no real title available?)
- scientific article; zbMATH DE number 1049042 (Why is no real title available?)
- scientific article; zbMATH DE number 7064043 (Why is no real title available?)
- Hybrid function spaces, heat and Navier-Stokes equations
- Multivariate approximation
- New error bounds for deep ReLU networks using sparse grids
- New explicit-in-dimension estimates for the cardinality of high-dimensional hyperbolic crosses and approximation of functions having mixed smoothness
- Optimal adaptive sampling recovery
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Optimized general sparse grid approximation spaces for operator equations
- Regularity and approximability of electronic wave functions
- Sampling and cubature on sparse grids based on a B-spline quasi-interpolation
- Sampling on energy-norm based sparse grids for the optimal recovery of Sobolev type functions in H^
- Sparse grids
- Tractability of multivariate problems. Volume I: Linear information
- Tractability of multivariate problems. Volume II: Standard information for functionals.
Cited in
(25)- Linearized two-layers neural networks in high dimension
- Constructive deep ReLU neural network approximation
- Approximation spaces of deep neural networks
- Exponential ReLU DNN expression of holomorphic maps in high dimension
- ReLU deep neural networks from the hierarchical basis perspective
- High-dimensional approximate r-nets
- scientific article; zbMATH DE number 6928881 (Why is no real title available?)
- Deep Neural Network Approximation Theory
- The Computational Complexity of ReLU Network Training Parameterized by Data Dimensionality
- Deep ReLU Networks Overcome the Curse of Dimensionality for Generalized Bandlimited Functions
- Learning Network from High-Dimensional Array Data
- Nonparametric regression on low-dimensional manifolds using deep ReLU networks: function approximation and statistical recovery
- Neural network approximation of continuous functions in high dimensions with applications to inverse problems
- Deep ReLU neural network approximation in Bochner spaces and applications to parametric PDEs
- On minimal representations of shallow ReLU networks
- Towards Lower Bounds on the Depth of ReLU Neural Networks
- Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class
- A multivariate Riesz basis of ReLU neural networks
- Collocation approximation by deep neural ReLU networks for parametric and stochastic PDEs with lognormal inputs
- Optimal approximation of infinite-dimensional holomorphic functions
- Approximation by tree tensor networks in high dimensions: Sobolev and compositional functions
- Approximation in shift-invariant spaces with deep ReLU neural networks
- Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks
- Sampling complexity of deep approximation spaces
- Solving PDEs on spheres with physics-informed convolutional neural networks
This page was built for publication: Deep ReLU neural networks in high-dimensional approximation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6079085)
