Representation formulas and pointwise properties for Barron functions
From MaRDI portal
(Redirected from Publication:2113295)
Special properties of functions of several variables, Hölder conditions, etc. (26B35) Representation and superposition of functions (26B40) Approximation by other special function classes (41A30) Banach spaces of continuous, differentiable or analytic functions (46E15) Artificial neural networks and deep learning (68T07)
Abstract: We study the natural function space for infinitely wide two-layer neural networks with ReLU activation (Barron space) and establish different representation formulae. In two cases, we describe the space explicitly up to isomorphism. Using a convenient representation, we study the pointwise properties of two-layer networks and show that functions whose singular set is fractal or curved (for example distance functions from smooth submanifolds) cannot be represented by infinitely wide two-layer networks with finite path-norm. We use this structure theorem to show that the only -diffeomorphisms which Barron space are affine. Furthermore, we show that every Barron function can be decomposed as the sum of a bounded and a positively one-homogeneous function and that there exist Barron functions which decay rapidly at infinity and are globally Lebesgue-integrable. This result suggests that two-layer neural networks may be able to approximate a greater variety of functions than commonly believed.
Recommendations
- A New Function Space from Barron Class and Application to Neural Network Approximation
- Understanding neural networks with reproducing kernel Banach spaces
- Provable approximation properties for deep neural networks
- Banach space representer theorems for neural networks and ridge splines
- A global universality of two-layer neural networks with ReLU activations
Cites work
- A mean field view of the landscape of two-layer neural networks
- A priori estimates of the population risk for two-layer neural networks
- Applied functional analysis. Functional analysis, Sobolev spaces and elliptic differential equations
- Approximation by Combinations of ReLU and Squared ReLU Ridge Functions With <inline-formula> <tex-math notation="LaTeX">$\ell^1$ </tex-math> </inline-formula> and <inline-formula>
- Approximation by superpositions of a sigmoidal function
- Breaking the curse of dimensionality with convex neural networks
- Functional analysis, Sobolev spaces and partial differential equations
- Hinging hyperplanes for regression, classification, and function approximation
- scientific article; zbMATH DE number 5263198 (Why is no real title available?)
- Kolmogorov width decay and poor approximators in machine learning: shallow neural networks, random feature models and neural tangent kernels
- Machine learning from a continuous viewpoint. I
- Mean field analysis of neural networks: a law of large numbers
- Measure theory and fine properties of functions
- The Variational Formulation of the Fokker--Planck Equation
- Universal approximation bounds for superpositions of a sigmoidal function
- Variational Analysis in Sobolev and BV Spaces
- Wahrscheinlichkeitstheorie
Cited in
(38)- Understanding neural networks with reproducing kernel Banach spaces
- The Barron space and the flow-induced function spaces for neural network models
- Uniform approximation rates and metric entropy of shallow neural networks
- Greedy training algorithms for neural networks and applications to PDEs
- Two-Layer Neural Networks with Values in a Banach Space
- Nonlinear weighted directed acyclic graph and a priori estimates for neural networks
- Deep Ritz method for the spectral fractional Laplacian equation using the Caffarelli-Silvestre extension
- A New Function Space from Barron Class and Application to Neural Network Approximation
- Piecewise linear functions representable with infinite width shallow ReLU neural networks
- Deep Neural Networks for Solving Large Linear Systems Arising from High-Dimensional Problems
- A convergent deep learning algorithm for approximation of polynomials
- A priori generalization error analysis of two-layer neural networks for solving high dimensional Schrödinger eigenvalue problems
- Neural network approximation and estimation of classifiers with classification boundary in a Barron class
- Two-layer networks with the \(\text{ReLU}^k\) activation function: Barron spaces and derivative approximation
- Low dimensional approximation and generalization of multivariate functions on smooth manifolds using deep ReLU neural networks
- Solving the regularized Schamel equation by the singular planar dynamical system method and the deep learning method
- Numerical analysis of physics-informed neural networks and related models in physics-informed machine learning
- Numerical solution of Poisson partial differential equation in high dimension using two-layer neural networks
- Embeddings between Barron spaces with higher-order activation functions
- Applied harmonic analysis and data science. Abstracts from the workshop held April 21--26, 2024
- Solution theory of Hamilton-Jacobi-Bellman equations in spectral Barron spaces
- Compositional function spaces for deep learning
- Bayesian scalar-on-image regression with a spatially varying single-layer neural network prior
- Understanding the training of infinitely deep and wide ResNets with conditional optimal transport
- Spectral Barron space for deep neural network approximation
- ReLU neural networks with linear layers are biased towards single- and multi-index models
- A duality framework for analyzing random feature and two-layer neural networks
- Learning a sparse representation of Barron functions with the inverse scale space flow
- Embedding inequalities for Barron-type spaces
- Information preservation with Wasserstein autoencoders: generation consistency and adversarial robustness
- Towards continuous mathematical models for the analysis of classes of deep neural networks
- Finite-difference least square methods for solving Hamilton-Jacobi equations using neural networks
- Multiscale neural networks for approximating Green's functions
- A Lipschitz spaces view of infinitely wide shallow neural networks
- Elliptic Regularity Theory in Barron Spaces and Applications to the Deep Ritz Method
- Curse of dimensionality in neural network optimization
- Barron space for graph convolutional neural networks
- Stable learning using spiking neural networks equipped with affine encoders and decoders
This page was built for publication: Representation formulas and pointwise properties for Barron functions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2113295)
