The Barron space and the flow-induced function spaces for neural network models
From MaRDI portal
(Redirected from Publication:2117337)
Abstract: One of the key issues in the analysis of machine learning models is to identify the appropriate function space and norm for the model. This is the set of functions endowed with a quantity which can control the approximation and estimation errors by a particular machine learning model. In this paper, we address this issue for two representative neural network models: the two-layer networks and the residual neural networks. We define the Barron space and show that it is the right space for two-layer neural network models in the sense that optimal direct and inverse approximation theorems hold for functions in the Barron space. For residual neural network models, we construct the so-called flow-induced function space, and prove direct and inverse approximation theorems for this space. In addition, we show that the Rademacher complexity for bounded sets under these norms has the optimal upper bounds.
Recommendations
- A New Function Space from Barron Class and Application to Neural Network Approximation
- Understanding neural networks with reproducing kernel Banach spaces
- Representation formulas and pointwise properties for Barron functions
- Approximation spaces of deep neural networks
- Rademacher complexity and the generalization error of residual networks
Cites work
- 10.1162/153244303321897690
- A priori estimates of the population risk for two-layer neural networks
- Approximation and estimation bounds for artificial neural networks
- Bounds on rates of variable-basis and neural-network approximation
- Breaking the curse of dimensionality with convex neural networks
- scientific article; zbMATH DE number 48727 (Why is no real title available?)
- scientific article; zbMATH DE number 477682 (Why is no real title available?)
- scientific article; zbMATH DE number 1972910 (Why is no real title available?)
- scientific article; zbMATH DE number 1785556 (Why is no real title available?)
- On the tractability of multivariate integration and approximation by neural networks
- Proof that deep artificial neural networks overcome the curse of dimensionality in the numerical approximation of Kolmogorov partial differential equations with constant diffusion and nonlinear drift coefficients
- The finite element methods for elliptic problems.
- Theory of Reproducing Kernels
- Understanding machine learning. From theory to algorithms
- Universal approximation bounds for superpositions of a sigmoidal function
Cited in
(85)- Understanding neural networks with reproducing kernel Banach spaces
- Nonconvex regularization for sparse neural networks
- Learning the mapping \(\mathbf{x}\mapsto \sum\limits_{i=1}^d x_i^2\): the cost of finding the needle in a haystack
- Active learning based sampling for high-dimensional nonlinear partial differential equations
- Low-rank kernel approximation of Lyapunov functions using neural networks
- Greedy training algorithms for neural networks and applications to PDEs
- Algorithms for solving high dimensional PDEs: from nonlinear Monte Carlo to machine learning
- Deep Adaptive Basis Galerkin Method for High-Dimensional Evolution Equations With Oscillatory Solutions
- Two-Layer Neural Networks with Values in a Banach Space
- Deep Ritz method for the spectral fractional Laplacian equation using the Caffarelli-Silvestre extension
- The discovery of dynamics via linear multistep methods and deep learning: error estimation
- Deep ReLU neural networks overcome the curse of dimensionality for partial integrodifferential equations
- A New Function Space from Barron Class and Application to Neural Network Approximation
- Simultaneous neural network approximation for smooth functions
- Deep Neural Networks for Solving Large Linear Systems Arising from High-Dimensional Problems
- Finite difference schemes for time-space fractional diffusion equations in one- and two-dimensions
- Deep learning methods for partial differential equations and related parameter identification problems
- A class of dimension-free metrics for the convergence of empirical measures
- A priori generalization error analysis of two-layer neural networks for solving high dimensional Schrödinger eigenvalue problems
- A finite difference scheme for the two-dimensional Gray-Scott equation with fractional Laplacian
- A two-branch symmetric domain adaptation neural network based on Ulam stability theory
- Improved Analysis of PINNs: Alleviate the CoD for Compositional Solutions
- Learning High-Dimensional McKean–Vlasov Forward-Backward Stochastic Differential Equations with General Distribution Dependence
- Control of neural transport for normalising flows
- Two-layer networks with the \(\text{ReLU}^k\) activation function: Barron spaces and derivative approximation
- Causal inference of general treatment effects using neural networks with a diverging number of confounders
- A Reduced Order Schwarz Method for Nonlinear Multiscale Elliptic Equations Based on Two-Layer Neural Networks
- A kernel framework for learning differential equations and their solution operators
- Efficient and stable SAV-based methods for gradient flows arising from deep learning
- Operator learning using random features: a tool for scientific computing
- Generalization error in the deep Ritz method with smooth activation functions
- Recovering the source term in elliptic equation via deep learning: method and convergence analysis
- Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning
- Numerical solution of Poisson partial differential equation in high dimension using two-layer neural networks
- Approximation results for gradient flow trained shallow neural networks in \(1d\)
- Weighted variation spaces and approximation by shallow ReLU networks
- Applied harmonic analysis and data science. Abstracts from the workshop held April 21--26, 2024
- Solution theory of Hamilton-Jacobi-Bellman equations in spectral Barron spaces
- Approximation and estimation capability of vision transformers for hierarchical compositional models
- Universal approximation of dynamical systems by semiautonomous neural ODEs and applications
- Weak minimization-sampling network for solving PDEs
- Representation of the solution of fractional Schrödinger equations with shallow neural network
- Approximation and gradient descent training with neural networks
- VE-cGAN: improved generalization analysis of conditional generative adversarial networks using vicinal estimation
- Local randomized neural networks with discontinuous Galerkin methods for KdV-type and Burgers equations
- Robust kernel-based gradient descent with random features
- A modified iterative PINN algorithm for strongly coupled system of boundary layer originated convection diffusion reaction problems in MHD flows with analysis
- Adaptive random Fourier features training stabilized by resampling with applications in image regression
- Bayesian scalar-on-image regression with a spatially varying single-layer neural network prior
- DRM revisited: a complete error analysis
- Sharp lower bounds on interpolation by deep ReLU neural networks at irregularly spaced data
- Layer separation deep learning model with auxiliary variables for partial differential equations
- On the rates of convergence for learning with convolutional neural networks
- Optimal approximation of zonoids and uniform approximation by shallow neural networks
- Understanding the training of infinitely deep and wide ResNets with conditional optimal transport
- Anti-symmetric Barron functions and their approximation with sums of determinants
- Spectral Barron space for deep neural network approximation
- ReLU neural networks with linear layers are biased towards single- and multi-index models
- A duality framework for analyzing random feature and two-layer neural networks
- Optimal transport map estimation in general function spaces
- Learning a sparse representation of Barron functions with the inverse scale space flow
- Approximation results for gradient flow trained neural networks
- Embedding inequalities for Barron-type spaces
- Reinforcement learning with function approximation: from linear to nonlinear
- RNN-attention based deep learning for solving inverse boundary problems in nonlinear Marshak waves
- Approximation of functionals by neural network without curse of dimensionality
- A priori estimates for deep residual network in continuous-time reinforcement learning
- A finite difference method for elliptic equations with the variable-order fractional derivative
- Duality for neural networks through reproducing kernel Banach spaces
- Deep learning for PDE-based inverse problems. Abstracts from the workshop held October 27 -- November 1, 2024
- Representation and regression problems in neural networks: relaxation, generalization, and numerics
- On understanding and overcoming spectral biases of deep neural network learning methods for solving PDEs
- Random projection neural networks of best approximation: convergence theory and practical applications
- Finite element operator network for solving elliptic-type parametric PDEs
- Towards continuous mathematical models for the analysis of classes of deep neural networks
- Compositional sparsity, approximation classes, and parametric transport equations
- An operator learning perspective on parameter-to-observable maps
- Multiscale neural networks for approximating Green's functions
- A simple and fast finite difference method for the integral fractional Laplacian of variable order
- Error analysis of the deep mixed residual method for high-order elliptic equations
- The parametric complexity of operator learning
- Universal approximation results for neural networks with non-polynomial activation function over non-compact domains
- A Lipschitz spaces view of infinitely wide shallow neural networks
- Time-frequency analysis for neural networks
- Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Method
This page was built for publication: The Barron space and the flow-induced function spaces for neural network models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2117337)
