Function approximation by deep neural networks with parameters \0, 12, 1,2\

DOI10.1007/S42519-021-00229-5MaRDI QIDQ2074650zbMATH OpenOpenAlexFDO

Authors Aleksandr Beknazaryan

Publication date 10 February 2022

Published in Journal of Statistical Theory and Practice (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/2103.08659

zbMATH Keywords

entropyfunction approximationneural networksnonparametric regression

Mathematics Subject Classification ID

Approximation by arbitrary nonlinear expressions; widths and entropy (41A46) General theory of stochastic processes (60G07) Nonparametric estimation (62G05) Nonparametric regression and quantile regression (62G08) Artificial neural networks and deep learning (68T07)

Abstract: In this paper it is shown that $$-smooth functions can be approximated by deep neural networks with ReLU activation function and with parameters $0,pmfrac12,pm1,2$. The $l_0$ and $l_1$ parameter norms of considered networks are thus equivalent. The depth, width and the number of active parameters of the constructed networks have, up to a logarithmic factor, the same dependence on the approximation error as the networks with parameters in $[-1,1]$. In particular, this means that the nonparametric regression estimation with the constructed networks attains the same convergence rate as with sparse networks with parameters in $[-1,1]$.

Recommendations

• Function approximation by deep networks
• On the approximation of rough functions with deep neural networks
• The gap between theory and practice in function approximation with deep neural networks
• Approximation of nonlinear functionals using deep ReLU networks
• Neural networks for optimal approximation of continuous functions in \(\mathbb R^d\)
• Function approximation by random neural networks with a bounded number of layers
• scientific article; zbMATH DE number 852149
• Approximating functions with multi-features by deep convolutional neural networks
• scientific article; zbMATH DE number 5839712

Cites work

• A distribution-free theory of nonparametric regression
• Convergence of stochastic processes
• Error bounds for approximations with deep ReLU networks
• Estimating a density under order restrictions: Nonasymptotic minimax risk
• Nonparametric regression using deep neural networks with ReLU activation function
• Statistical guarantees for regularized neural networks

Cited in

(9)

• On the rate of convergence of fully connected deep neural network regression estimates
• Nonparametric regression with modified ReLU networks
• Depth separations in neural networks: what is actually being separated?
• Nonlinear approximation via compositions
• Function approximation by deep networks
• Nonparametric regression using deep neural networks with ReLU activation function
• A note on the applications of one primary function in deep neural networks
• On decision regions of narrow deep neural networks
• Simultaneous approximation of a smooth function and its derivatives by deep neural networks with piecewise-polynomial activations

This page was built for publication: Function approximation by deep neural networks with parameters \(\{0, \pm \frac{1}{2}, \pm 1,2\}\)

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2074650)