Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions

DOI10.1016/J.JCO.2018.09.002MaRDI QIDQ1734693zbMATH OpenOpenAlexWikidataFDO

Authors H. N. Mhaskar

Publication date 27 March 2019

Published in Journal of Complexity (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1709.08174

zbMATH Keywords

approximation on Euclidean sphereneural network approximationrelu (rectified linear unit) functions

Mathematics Subject Classification ID

Approximation by other special function classes (41A30)

Abstract: A zonal function (ZF) network on the $q$ dimensional sphere $mathbb{S}^q$ is a network of the form $mathbfxmapstosu{m}_{k=1}^n{a}_{k}phi(mathbfxcdotmathbf{x}_k)$ where $phi:[-1,1]omathbfR$ is the activation function, $mathbf{x}_{k}inmathbb{S}^q$ are the centers, and $a_{k}inmathbbR$. While the approximation properties of such networks are well studied in the context of positive definite activation functions, recent interest in deep and shallow networks motivate the study of activation functions of the form $phi(t)=|t|$, which are not positive definite. In this paper, we define an appropriate smoothess class and establish approximation properties of such networks for functions in this class. The centers can be chosen independently of the target function, and the coefficients are linear combinations of the training data. The constructions preserve rotational symmetries.

Recommendations

• Approximation by spherical neural networks with zonal functions
• Approximation properties of zonal function networks using scattered data on the sphere
• Approximation by spherical neural networks with sigmoidal functions
• Weighted quadrature formulas and approximation by zonal function networks on the sphere
• Construction and approximation of spherical neural networks

Cites work

• Approximation by superposition of sigmoidal and radial basis functions
• Approximation by superpositions of a sigmoidal function
• Approximation properties of zonal function networks using scattered data on the sphere
• Deep vs. shallow networks: an approximation theory perspective
• Dimension-independent bounds on the degree of approximation by neural networks
• Eignets for function approximation on manifolds
• scientific article; zbMATH DE number 3474264 (Why is no real title available?)
• scientific article; zbMATH DE number 3536702 (Why is no real title available?)
• scientific article; zbMATH DE number 1977310 (Why is no real title available?)
• scientific article; zbMATH DE number 1405266 (Why is no real title available?)
• scientific article; zbMATH DE number 3314813 (Why is no real title available?)
• scientific article; zbMATH DE number 3108589 (Why is no real title available?)
• Localized linear polynomial operators and quadrature formulas on the sphere
• Marcinkiewicz-Zygmund measures on manifolds
• On the representation of smooth functions on the sphere using finitely many bits
• On the tractability of multivariate integration and approximation by neural networks
• Polynomial operators and local approximation of solutions of pseudo-differential equations on the sphere
• Polynomial operators and local smoothness classes on the unit interval
• Universal approximation bounds for superpositions of a sigmoidal function
• Weighted quadrature formulas and approximation by zonal function networks on the sphere

Cited in

(8)

• Approximation properties of zonal function networks using scattered data on the sphere
• An analysis of training and generalization errors in shallow and deep networks
• Function approximation by deep networks
• Local approximation of operators
• Activation Functions Defined on Higher-Dimensional Spaces for Approximation on Compact Sets with and without Scaling
• Rates of approximation by ReLU shallow neural networks
• Optimal approximation of zonoids and uniform approximation by shallow neural networks
• Approximation by non-symmetric networks for cross-domain learning

This page was built for publication: Function approximation with zonal function networks with activation functions analogous to the rectified linear unit functions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1734693)