The Barron space and the flow-induced function spaces for neural network models
From MaRDI portal
(Redirected from Publication:2117337)
Abstract: One of the key issues in the analysis of machine learning models is to identify the appropriate function space and norm for the model. This is the set of functions endowed with a quantity which can control the approximation and estimation errors by a particular machine learning model. In this paper, we address this issue for two representative neural network models: the two-layer networks and the residual neural networks. We define the Barron space and show that it is the right space for two-layer neural network models in the sense that optimal direct and inverse approximation theorems hold for functions in the Barron space. For residual neural network models, we construct the so-called flow-induced function space, and prove direct and inverse approximation theorems for this space. In addition, we show that the Rademacher complexity for bounded sets under these norms has the optimal upper bounds.
Recommendations
- A New Function Space from Barron Class and Application to Neural Network Approximation
- Understanding neural networks with reproducing kernel Banach spaces
- Representation formulas and pointwise properties for Barron functions
- Approximation spaces of deep neural networks
- Rademacher complexity and the generalization error of residual networks
Cites work
- scientific article; zbMATH DE number 48727 (Why is no real title available?)
- scientific article; zbMATH DE number 477682 (Why is no real title available?)
- scientific article; zbMATH DE number 1972910 (Why is no real title available?)
- scientific article; zbMATH DE number 1785556 (Why is no real title available?)
- 10.1162/153244303321897690
- A priori estimates of the population risk for two-layer neural networks
- Approximation and estimation bounds for artificial neural networks
- Bounds on rates of variable-basis and neural-network approximation
- Breaking the curse of dimensionality with convex neural networks
- On the tractability of multivariate integration and approximation by neural networks
- Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
- The finite element methods for elliptic problems.
- Theory of Reproducing Kernels
- Understanding machine learning. From theory to algorithms
- Universal approximation bounds for superpositions of a sigmoidal function
Cited in
(86)- Spectral Barron space for deep neural network approximation
- ReLU neural networks with linear layers are biased towards single- and multi-index models
- A duality framework for analyzing random feature and two-layer neural networks
- Optimal transport map estimation in general function spaces
- Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Method
- A two-branch symmetric domain adaptation neural network based on Ulam stability theory
- On understanding and overcoming spectral biases of deep neural network learning methods for solving PDEs
- Learning a sparse representation of Barron functions with the inverse scale space flow
- Approximation results for gradient flow trained neural networks
- Embedding inequalities for Barron-type spaces
- Reinforcement learning with function approximation: from linear to nonlinear
- RNN-attention based deep learning for solving inverse boundary problems in nonlinear Marshak waves
- Approximation of functionals by neural network without curse of dimensionality
- A priori estimates for deep residual network in continuous-time reinforcement learning
- Deep learning methods for partial differential equations and related parameter identification problems
- Understanding neural networks with reproducing kernel Banach spaces
- A finite difference method for elliptic equations with the variable-order fractional derivative
- Deep Neural Networks for Solving Large Linear Systems Arising from High-Dimensional Problems
- Duality for neural networks through reproducing kernel Banach spaces
- Two-Layer Neural Networks with Values in a Banach Space
- The discovery of dynamics via linear multistep methods and deep learning: error estimation
- Deep learning for PDE-based inverse problems. Abstracts from the workshop held October 27 -- November 1, 2024
- Representation and regression problems in neural networks: relaxation, generalization, and numerics
- Nonconvex regularization for sparse neural networks
- Improved Analysis of PINNs: Alleviate the CoD for Compositional Solutions
- Learning the mapping \(\mathbf{x}\mapsto \sum\limits_{i=1}^d x_i^2\): the cost of finding the needle in a haystack
- Random projection neural networks of best approximation: convergence theory and practical applications
- Operator learning using random features: a tool for scientific computing
- Low-rank kernel approximation of Lyapunov functions using neural networks
- Generalization error in the deep Ritz method with smooth activation functions
- Recovering the source term in elliptic equation via deep learning: method and convergence analysis
- Finite element operator network for solving elliptic-type parametric PDEs
- Towards continuous mathematical models for the analysis of classes of deep neural networks
- A simple and fast finite difference method for the integral fractional Laplacian of variable order
- Compositional sparsity, approximation classes, and parametric transport equations
- Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning
- A New Function Space from Barron Class and Application to Neural Network Approximation
- An operator learning perspective on parameter-to-observable maps
- Finite difference schemes for time-space fractional diffusion equations in one- and two-dimensions
- Multiscale neural networks for approximating Green's functions
- Error analysis of the deep mixed residual method for high-order elliptic equations
- A priori generalization error analysis of two-layer neural networks for solving high dimensional Schrödinger eigenvalue problems
- Deep Ritz method for the spectral fractional Laplacian equation using the Caffarelli-Silvestre extension
- Learning High-Dimensional McKean–Vlasov Forward-Backward Stochastic Differential Equations with General Distribution Dependence
- The parametric complexity of operator learning
- Control of neural transport for normalising flows
- A class of dimension-free metrics for the convergence of empirical measures
- Two-layer networks with the \(\text{ReLU}^k\) activation function: Barron spaces and derivative approximation
- Causal inference of general treatment effects using neural networks with a diverging number of confounders
- Numerical solution of Poisson partial differential equation in high dimension using two-layer neural networks
- Solution theory of Hamilton-Jacobi-Bellman equations in spectral Barron spaces
- Universal approximation results for neural networks with non-polynomial activation function over non-compact domains
- A Reduced Order Schwarz Method for Nonlinear Multiscale Elliptic Equations Based on Two-Layer Neural Networks
- A finite difference scheme for the two-dimensional Gray-Scott equation with fractional Laplacian
- Approximation and estimation capability of vision transformers for hierarchical compositional models
- Universal approximation of dynamical systems by semiautonomous neural ODEs and applications
- Weak minimization-sampling network for solving PDEs
- A Lipschitz spaces view of infinitely wide shallow neural networks
- Representation of the solution of fractional Schrödinger equations with shallow neural network
- Approximation and gradient descent training with neural networks
- VE-cGAN: improved generalization analysis of conditional generative adversarial networks using vicinal estimation
- Deep Adaptive Basis Galerkin Method for High-Dimensional Evolution Equations With Oscillatory Solutions
- Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
- Local randomized neural networks with discontinuous Galerkin methods for KdV-type and Burgers equations
- Active learning based sampling for high-dimensional nonlinear partial differential equations
- Robust kernel-based gradient descent with random features
- A modified iterative PINN algorithm for strongly coupled system of boundary layer originated convection diffusion reaction problems in MHD flows with analysis
- Approximation results for gradient flow trained shallow neural networks in \(1d\)
- Adaptive random Fourier features training stabilized by resampling with applications in image regression
- Bayesian scalar-on-image regression with a spatially varying single-layer neural network prior
- DRM revisited: a complete error analysis
- Weighted variation spaces and approximation by shallow ReLU networks
- Deep ReLU neural networks overcome the curse of dimensionality for partial integrodifferential equations
- Anti-symmetric Barron functions and their approximation with sums of determinants
- Sharp lower bounds on interpolation by deep ReLU neural networks at irregularly spaced data
- Layer separation deep learning model with auxiliary variables for partial differential equations
- A kernel framework for learning differential equations and their solution operators
- Efficient and stable SAV-based methods for gradient flows arising from deep learning
- On the rates of convergence for learning with convolutional neural networks
- Time-frequency analysis for neural networks
- Simultaneous neural network approximation for smooth functions
- Applied harmonic analysis and data science. Abstracts from the workshop held April 21--26, 2024
- Optimal approximation of zonoids and uniform approximation by shallow neural networks
- Greedy training algorithms for neural networks and applications to PDEs
- Understanding the training of infinitely deep and wide ResNets with conditional optimal transport
- Sharp Sobolev Approximation on General Domains by Linearized Shallow Networks with Analytic Activations
This page was built for publication: The Barron space and the flow-induced function spaces for neural network models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2117337)
