Error bounds for approximations with deep ReLU neural networks in W^s , p norms
From MaRDI portal
Publication:5132228
Rate of convergence, degree of approximation (41A25) Sobolev spaces and other spaces of ``smooth functions, embedding theorems, trace theorems (46E35) Finite element, Rayleigh-Ritz and Galerkin methods for boundary value problems involving PDEs (65N30) Learning and adaptive systems in artificial intelligence (68T05)
Abstract: We analyze approximation rates of deep ReLU neural networks for Sobolev-regular functions with respect to weaker Sobolev norms. First, we construct, based on a calculus of ReLU networks, artificial neural networks with ReLU activation functions that achieve certain approximation rates. Second, we establish lower bounds for the approximation by ReLU neural networks for classes of Sobolev-regular functions. Our results extend recent advances in the approximation theory of ReLU networks to the regime that is most relevant for applications in the numerical analysis of partial differential equations.
Recommendations
- Error bounds for approximations with deep ReLU networks
- New error bounds for deep ReLU networks using sparse grids
- Deep ReLU networks and high-order finite element methods
- Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
- Nonlinear approximation and (deep) ReLU networks
Cites work
- An introduction to Sobolev spaces and interpolation spaces
- Approximation and estimation bounds for artificial neural networks
- Approximation by superpositions of a sigmoidal function
- Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- Deep neural networks for rotation-invariance approximation and learning
- DGM: a deep learning algorithm for solving partial differential equations
- Elliptic partial differential equations of second order
- Error bounds for approximations with deep ReLU networks
- Functional analysis, Sobolev spaces and partial differential equations
- Group invariant scattering
- scientific article; zbMATH DE number 4138299 (Why is no real title available?)
- scientific article; zbMATH DE number 3678487 (Why is no real title available?)
- scientific article; zbMATH DE number 3491650 (Why is no real title available?)
- scientific article; zbMATH DE number 3536702 (Why is no real title available?)
- scientific article; zbMATH DE number 3602126 (Why is no real title available?)
- scientific article; zbMATH DE number 1237602 (Why is no real title available?)
- scientific article; zbMATH DE number 3447208 (Why is no real title available?)
- scientific article; zbMATH DE number 2219854 (Why is no real title available?)
- Interpolation theory
- Learning Theory
- Neural Network Learning
- Nonlinear partial differential equations with applications
- Nonlocal diffusion and applications
- On the degree of nonlinear spline approximation in Besov-Sobolev spaces
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Optimal approximation with sparsely connected deep neural networks
- Provable approximation properties for deep neural networks
- Solving high-dimensional partial differential equations using deep learning
- The deep Ritz method: a deep learning-based numerical algorithm for solving variational problems
- The Mathematical Theory of Finite Element Methods
Cited in
(99)- Efficient approximation of solutions of parametric linear transport equations by ReLU DNNs
- Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
- Fast generalization error bound of deep learning without scale invariance of activation functions
- Approximation rates for neural networks with encodable weights in smoothness spaces
- Solving PDEs by variational physics-informed neural networks: an a posteriori error analysis
- Error analysis for physics-informed neural networks (PINNs) approximating Kolmogorov PDEs
- Error bounds for ReLU networks with depth and width parameters
- A theoretical analysis of deep neural networks and parametric PDEs
- Adaptive two-layer ReLU neural network. I: Best least-squares approximation
- On sharpness of an error bound for deep ReLU network approximation
- Deep learning for the partially linear Cox model
- Approximation properties of deep ReLU CNNs
- ReLU deep neural networks from the hierarchical basis perspective
- Variational physics informed neural networks: the role of quadratures and test functions
- Error bounds for approximations with deep ReLU networks
- Nonasymptotic bounds on the \(L_{2}\) error of neural network regression estimates
- Neural control of discrete weak formulations: Galerkin, least squares \& minimal-residual methods with quasi-optimal weights
- DeepBND: a machine learning approach to enhance multiscale solid mechanics
- Growing axons: greedy learning of neural networks with application to function approximation
- Learning elliptic partial differential equations with randomized linear algebra
- Sobolev-type embeddings for neural network approximation spaces
- Approximation by Combinations of ReLU and Squared ReLU Ridge Functions With <inline-formula> <tex-math notation="LaTeX">$\ell^1$ </tex-math> </inline-formula> and <inline-formula> <tex-math notation="LaTeX">$\ell^0$ </tex-math> </inline-formula> Controls
- The gap between theory and practice in function approximation with deep neural networks
- Deep network with approximation error being reciprocal of width to power of square root of depth
- Optimization with learning-informed differential equation constraints and its applications
- New error bounds for deep ReLU networks using sparse grids
- A deep learning approach to Reduced Order Modelling of parameter dependent partial differential equations
- Error Analysis and Improving the Accuracy of Winograd Convolution for Deep Neural Networks
- Approximations with deep neural networks in Sobolev time-space
- Convergence rate analysis for deep Ritz method
- scientific article; zbMATH DE number 7546040 (Why is no real title available?)
- Full error analysis for the training of deep neural networks
- A mesh-less, ray-based deep neural network method for the Helmholtz equation with high frequency
- Deep ReLU networks and high-order finite element methods
- Deep Network Approximation for Smooth Functions
- Deep Nitsche Method: Deep Ritz Method with Essential Boundary Conditions
- Non-Intrusive Reduced Order Modeling of Convection Dominated Flows Using Artificial Neural Networks with Application to Rayleigh-Taylor Instability
- A Framework for the Construction of Upper Bounds on the Number of Affine Linear Regions of ReLU Feed-Forward Neural Networks
- Approximation bounds for norm constrained neural networks with applications to regression and GANs
- Simultaneous neural network approximation for smooth functions
- On the approximation of functions by tanh neural networks
- Deep ReLU neural network approximation in Bochner spaces and applications to parametric PDEs
- Rates of approximation by ReLU shallow neural networks
- Deep learning methods for partial differential equations and related parameter identification problems
- Some elliptic second order problems and neural network solutions: existence and error estimates
- On decision regions of narrow deep neural networks
- Deep ReLU neural networks in high-dimensional approximation
- Towards Lower Bounds on the Depth of ReLU Neural Networks
- Exponential ReLU neural network approximation rates for point and edge singularities
- Friedrichs Learning: Weak Solutions of Partial Differential Equations via Deep Learning
- Neural ODE Control for Classification, Approximation, and Transport
- SignReLU neural network and its approximation ability
- Deep Neural Networks with ReLU-Sine-Exponential Activations Break Curse of Dimensionality in Approximation on Hölder Class
- DPK: Deep Neural Network Approximation of the First Piola-Kirchhoff Stress
- A Novel Deep Neural Network Algorithm for the Helmholtz Scattering Problem In the Unbounded Domain
- Collocation approximation by deep neural ReLU networks for parametric and stochastic PDEs with lognormal inputs
- Deep learning via dynamical systems: an approximation perspective
- Neural Control of Parametric Solutions for High-Dimensional Evolution PDEs
- Approximation in shift-invariant spaces with deep ReLU neural networks
- Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations
- Solving PDEs on unknown manifolds with machine learning
- Neural network expression rates and applications of the deep parametric PDE method in counterparty credit risk
- An optimal control framework for adaptive neural ODEs
- Deep neural network expressivity for optimal stopping problems
- A descent algorithm for the optimal control of ReLU neural network informed PDEs based on approximate directional derivatives
- Deep ReLU networks and high-order finite element methods. II: Chebyšev emulation
- Inf-sup neural networks for high-dimensional elliptic PDE problems
- Error analysis for deep neural network approximations of parametric hyperbolic conservation laws
- Approximation rates for deep calibration of (rough) stochastic volatility models
- Extending error bounds for radial basis function interpolation to measuring the error in higher order Sobolev norms
- Approximation results for gradient flow trained shallow neural networks in \(1d\)
- Importance sampling for option pricing with feedforward neural networks
- WAN discretization of PDEs: best approximation, stabilization, and essential boundary conditions
- A hybrid physics-informed neural network based multiscale solver as a partial differential equation constrained optimization problem
- Generalization error analysis of deep backward dynamic programming for solving nonlinear PDEs
- Representation of the solution of fractional Schrödinger equations with shallow neural network
- Approximation and gradient descent training with neural networks
- DRM revisited: a complete error analysis
- Analysis of structured deep kernel networks
- Why shallow networks struggle to approximate and learn high frequencies
- Operator theory, kernels, and feedforward neural networks
- Structured and balanced multicomponent and multilayer neural networks
- On the optimal approximation of Sobolev and Besov functions using deep ReLU neural networks
- Approximation results for gradient flow trained neural networks
- First-Order Conditions for the Optimal Control of Learning-Informed Nonsmooth PDEs
- Combining physics-based and data-driven models: advancing the frontiers of research with scientific machine learning
- Random projection neural networks of best approximation: convergence theory and practical applications
- Some thoughts on compositional tensor networks
- A certified wavelet-based physics-informed neural network for the solution of parameterized partial differential equations
- Compositional sparsity, approximation classes, and parametric transport equations
- Deep Ritz--finite element methods: neural network methods trained with finite elements
- Residual multi-fidelity neural network computing
- Interpolation, approximation, and controllability of deep neural networks
- Variable Significance Testing for the Deep Cox Model
- Deep orthogonal decomposition: a continuously adaptive neural network approach to model order reduction of parametrized partial differential equations
- Approximation of Analytic Functions by ReLU Neural Networks with Adjustable Depth and Width
- Distribution estimation via flow matching with Lipschitz guarantees
- Deep neural networks are adaptive to function regularity and data distribution in approximation and estimation
- A modified gradient descent strategy for computing neural networks solutions
This page was built for publication: Error bounds for approximations with deep ReLU neural networks in \(W^{s , p}\) norms
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5132228)
