A multivariate Riesz basis of ReLU neural networks
Artificial neural networks have in the last 10 years been responsible for the enormous success in answering many questions in approximation theory and other learning tasks in a vast number of scientific areas, for example computer vision, speech recognition, natural language processing, game theory, signal processing, neuroscience, social sciences and many others. This paper contributes to a theoretical framework for understanding neural networks in the following sense. An extremely important property of artificial neural networks is that they can approximate extremely well functions of many variables, which often allows us to avoid the curse of dimensionality. The curse of dimensionality refers to various phenomena that arise when analyzing and organizing data in high-dimensional spaces that do not occur in low-dimensional settings such as the three-dimensional physical space of everyday experience. The expression was coined by Richard E. Bellman when considering problems in dynamic programming. The curse generally refers to issues that arise when the number of datapoints is small (in a suitably defined sense) relative to the intrinsic dimension of the data. The aim of the work in this paper is to understand the possibility to use artificial neural networks for the approximation of multivariate functions by constructing a new system of ReLU neural networks which forms a Riesz basis of the space \(L_2([0, 1]^d)\) for every \(d\geq 1\). In order to do this, the authors consider the trigonometric-like system of piecewise linear functions introduced recently by Daubechies, DeVore, Foucart, Hanin, and Petrova and use Gershgorin's theorem to prove that these indeed provide a Riesz basis of \(L_2([0, 1])\). The authors then generalize this system to large dimensions \(d>1\) and do this by avoiding tensor products. The paper is well written with a good set of references.
- ReLU deep neural networks from the hierarchical basis perspective
- New error bounds for deep ReLU networks using sparse grids
- Nonlinear approximation and (deep) ReLU networks
- Approximation by ridge functions and neural networks with a bounded number of neurons
- A note on the expressive power of deep rectified linear unit networks in high-dimensional spaces
- An introduction to frames and Riesz bases
- Approximation spaces of deep neural networks
- Deep Neural Network Approximation Theory
- Deep ReLU neural networks in high-dimensional approximation
- DeepStack: expert-level artificial intelligence in heads-up no-limit poker
- Error bounds for approximations with deep ReLU networks
- Foundations of time-frequency analysis
- Frames and the projection method
- Frames containing a Riesz basis and approximation of the frame coefficients using finite-dimensional methods
- scientific article; zbMATH DE number 3968684 (Why is no real title available?)
- scientific article; zbMATH DE number 3776165 (Why is no real title available?)
- Matrix Analysis
- Neural network approximation
- New characterizations of Riesz bases
- Nonlinear approximation and (deep) ReLU networks
- Nonparametric regression using deep neural networks with ReLU activation function
- Optimal approximation of piecewise smooth functions using deep ReLU neural networks
- Optimal approximation with sparsely connected deep neural networks
- The tensor product of frames
This page was built for publication: A multivariate Riesz basis of ReLU neural networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6144893)
