Integral representations of shallow neural network with Rectified Power Unit activation function

DOI10.1016/J.NEUNET.2022.09.005MaRDI QIDQ6386309zbMATH OpenWikidataFDO

Authors Ahmed Abdeljawad, Philipp Grohs

Publication date 20 December 2021

Full work available at URL https://doi.org/10.1016/J.NEUNET.2022.09.005

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: In this effort, we derive a formula for the integral representation of a shallow neural network with the Rectified Power Unit activation function. Mainly, our first result deals with the univariate case of representation capability of RePU shallow networks. The multidimensional result in this paper characterizes the set of functions that can be represented with bounded norm and possibly unbounded width.

Recommendations

• Better approximations of high dimensional smooth functions by deep neural networks with rectified power units
• PowerNet: efficient representations of polynomials and smooth functions by deep neural networks with rectified power units
• Embeddings between Barron spaces with higher-order activation functions
• Approximation of smoothness classes by deep rectifier networks
• A representer theorem for deep neural networks

Cites work

• A course in analysis. Volume IV: Fourier analysis, ordinary differential equations, calculus of variations
• A Sobolev-type upper bound for rates of approximation by linear combinations of Heaviside plane waves
• Asymptotic formulas for the dual Radon transform and applications
• Banach space representer theorems for neural networks and ridge splines
• Complexity estimates based on integral transforms induced by computational units
• Deep Neural Network Approximation Theory
• Error bounds for approximations with deep ReLU networks
• Harmonic analysis of neural networks
• scientific article; zbMATH DE number 1405266 (Why is no real title available?)
• Integral combinations of Heavisides
• Measure theory. Vol. I and II
• Multilayer feedforward networks are universal approximators
• New range theorems for the dual Radon transform
• Nonparametric regression using deep neural networks with ReLU activation function
• Optimal approximation with sparsely connected deep neural networks
• Sparsest solutions of underdetermined linear systems via \( \ell _q\)-minimization for \(0<q\leqslant 1\)
• Support theorems for the Radon transform and Cramér-Wold theorems
• The Radon transform.
• Universal approximation bounds for superpositions of a sigmoidal function

Cited in

(2)

• Two-layer networks with the \(\text{ReLU}^k\) activation function: Barron spaces and derivative approximation
• Embeddings between Barron spaces with higher-order activation functions

This page was built for publication: Integral representations of shallow neural network with Rectified Power Unit activation function

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6386309)