On the rate of convergence of fully connected deep neural network regression estimates
From MaRDI portal
Publication:2054491
Abstract: Recent results in nonparametric regression show that deep learning, i.e., neural network estimates with many hidden layers, are able to circumvent the so-called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. One key feature of the neural networks used in these results is that their network architecture has a further constraint, namely the network sparsity. In this paper we show that we can get similar results also for least squares estimates based on simple fully connected neural networks with ReLU activation functions. Here either the number of neurons per hidden layer is fixed and the number of hidden layers tends to infinity suitably fast for sample size tending to infinity, or the number of hidden layers is bounded by some logarithmic factor in the sample size and the number of neurons per hidden layer tends to infinity suitably fast for sample size tending to infinity. The proof is based on new approximation results concerning deep neural networks.
Recommendations
- Analysis of the rate of convergence of fully connected deep neural network regression estimates with smooth activation function
- Estimation of a regression function on a manifold by fully connected deep neural networks
- On deep learning as a remedy for the curse of dimensionality in nonparametric regression
- Nonparametric regression using deep neural networks with ReLU activation function
- Function approximation by deep neural networks with parameters \(\{0, \pm \frac{1}{2}, \pm 1,2\}\)
Cites work
- scientific article; zbMATH DE number 17222 (Why is no real title available?)
- scientific article; zbMATH DE number 893887 (Why is no real title available?)
- A distribution-free theory of nonparametric regression
- Adaptive regression estimation with multilayer feedforward neural networks
- Additive regression and other nonparametric models
- Approximation and estimation bounds for artificial neural networks
- Deep Network Approximation for Smooth Functions
- Deep Neural Network Approximation Theory
- Deep vs. shallow networks: an approximation theory perspective
- Distribution-free consistency results in nonparametric discrimination and regression function estimation
- Investigating Smooth Multiple Regression by the Method of Average Derivatives
- Nonparametric Regression Based on Hierarchical Interaction Models
- Nonparametric regression using deep neural networks with ReLU activation function
- On deep learning as a remedy for the curse of dimensionality in nonparametric regression
- Optimal global rates of convergence for nonparametric regression
- Optimal smoothing in single-index models
- Penalized Spline Estimation for Partially Linear Single-Index Models
- Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
- The use of polynomial splines and their tensor products in multivariate function estimation. (With discussion)
- Universal approximation bounds for superpositions of a sigmoidal function
- Variable selection for the single-index model
Cited in
(38)- Applied harmonic analysis and data science. Abstracts from the workshop held April 21--26, 2024
- Asymptotic properties of neural network sieve estimators
- Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression
- Robust nonparametric regression based on deep ReLU neural networks
- Breaking the curse of dimensionality with convex neural networks
- Optimal convergence rates of deep neural networks in a classification setting
- Calibrating multi-dimensional complex ODE from noisy data via deep neural networks
- A deep learning method for pricing high-dimensional American-style options via state-space partition
- Fast convergence rates of deep neural networks for classification
- On the rate of convergence of image classifiers based on convolutional neural networks
- Analysis of the rate of convergence of two regression estimates defined by neural features which are easy to implement
- Intrinsic and extrinsic deep learning on manifolds
- Deep learning based on randomized quasi-Monte Carlo method for solving linear Kolmogorov partial differential equation
- Convergence analysis for over-parameterized deep learning
- Recovering the source term in elliptic equation via deep learning: method and convergence analysis
- Mini-workshop: Nonlinear approximation of high-dimensional functions in scientific computing. Abstracts from the mini-workshop held October 15--20, 2023
- Sampling complexity of deep approximation spaces
- Layer sparsity in neural networks
- Statistical theory for image classification using deep convolutional neural network with cross-entropy loss under the hierarchical max-pooling model
- Nonparametric regression using deep neural networks with ReLU activation function
- A Deep Generative Approach to Conditional Sampling
- Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local pooling
- Local convergence rates of the nonparametric least squares estimator with applications to transfer learning
- Analysis of the rate of convergence of fully connected deep neural network regression estimates with smooth activation function
- Deep neural networks for estimation and inference
- Adaptive deep learning for nonlinear time series models
- Convergence rates of deep ReLU networks for multiclass classification
- Convergence rates for shallow neural networks learned by gradient descent
- Asymptotic properties of one-layer artificial neural networks with sparse connectivity
- On deep learning as a remedy for the curse of dimensionality in nonparametric regression
- How do noise tails impact on deep ReLU networks?
- On the rate of convergence of a deep recurrent neural network estimate in a regression problem with dependent data
- Estimation of a regression function on a manifold by fully connected deep neural networks
- Deep nonparametric regression on approximate manifolds: nonasymptotic error bounds with polynomial prefactors
- Error analysis for empirical risk minimization over clipped ReLU networks in solving linear Kolmogorov partial differential equations
- Adaptive variational Bayes: optimality, computation and applications
- Asset pricing with neural networks: significance tests
- An error analysis for deep binary classification with sigmoid loss
This page was built for publication: On the rate of convergence of fully connected deep neural network regression estimates
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2054491)
