A global universality of two-layer neural networks with ReLU activations

DOI10.1155/2021/6637220MaRDI QIDQ2236407zbMATH OpenOpenAlexFDO

Authors Naoya Hatano, Masahiro Ikeda, Isao Ishikawa, Yoshihiro Sawano

Publication date 22 October 2021

Published in Journal of Function Spaces (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2011.10225

Mathematics Subject Classification ID

Approximation by other special function classes (41A30) Networks and circuits as models of computation; circuit complexity (68Q06) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: In the present study, we investigate a universality of neural networks, which concerns a density of the set of two-layer neural networks in a function spaces. There are many works that handle the convergence over compact sets. In the present paper, we consider a global convergence by introducing a norm suitably, so that our results will be uniform over any compact set.

Review text

Summary: In the present study, we investigate a universality of neural networks, which concerns a density of the set of two-layer neural networks in function spaces. There are many works that handle the convergence over compact sets. In the present paper, we consider a global convergence by introducing a norm suitably, so that our results will be uniform over any compact set.

Recommendations

• Neural network with unbounded activation functions is universal approximator
• Approximation capabilities of neural networks on unbounded domains
• Nonlinear approximation and (deep) ReLU networks
• Error bounds for approximations with deep ReLU networks
• Approximation properties of deep ReLU CNNs

Cites work

• Approximation by superpositions of a sigmoidal function
• scientific article; zbMATH DE number 1405266 (Why is no real title available?)
• Multilayer feedforward networks are universal approximators
• Neural network with unbounded activation functions is universal approximator

Cited in

(7)

• Topological properties of the set of functions generated by neural networks of fixed size
• Representation formulas and pointwise properties for Barron functions
• Heaviside function as an activation function
• Universality of neural networks with a sigmoidal activation or discriminatory functions on functional Banach lattices over the real line
• Concentration of measure and global optimization of Bayesian multilayer perceptron. I
• Elementary proof of Funahashi's theorem
• A unified and constructive framework for the universality of neural networks

This page was built for publication: A global universality of two-layer neural networks with ReLU activations

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2236407)