ReLU networks are universal approximators via piecewise linear or constant functions

DOI10.1162/NECO_A_01316MaRDI QIDQ3386431zbMATH OpenOpenAlexWikidataFDO

Authors Changcun Huang

Publication date 4 January 2021

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1162/neco_a_01316

Mathematics Subject Classification ID

Rate of convergence, degree of approximation (41A25) Approximation by other special function classes (41A30) Multidimensional problems (41A63) Artificial neural networks and deep learning (68T07)

Recommendations

• Nonlinear approximation and (deep) ReLU networks
• Approximation properties of deep ReLU CNNs
• Neural network with unbounded activation functions is universal approximator
• Error bounds for deep ReLU networks using the Kolmogorov-Arnold superposition theorem
• Error bounds for approximations with deep ReLU networks

Cites work

• Approximation by superpositions of a sigmoidal function
• Computational geometry. Algorithms and applications.
• Error bounds for approximations with deep ReLU networks
• Multilayer feedforward networks are universal approximators
• Why does deep and cheap learning work so well?

Cited in

(6)

• A class of universal approximators of real continuous functions revisited
• Neural network with unbounded activation functions is universal approximator
• Wide neural networks of any depth evolve as linear models under gradient descent ^*
• SignReLU neural network and its approximation ability
• Invariant set estimation for piecewise affine dynamical systems using piecewise affine barrier function
• On neural network approximation of ideal adversarial attack and convergence of adversarial training

This page was built for publication: ReLU networks are universal approximators via piecewise linear or constant functions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3386431)