Statistical guarantees for regularized neural networks
From MaRDI portal
Abstract: Neural networks have become standard tools in the analysis of data, but they lack comprehensive mathematical theories. For example, there are very few statistical guarantees for learning neural networks from data, especially for classes of estimators that are used in practice or at least similar to such. In this paper, we develop a general statistical guarantee for estimators that consist of a least-squares term and a regularizer. We then exemplify this guarantee with -regularization, showing that the corresponding prediction error increases at most sub-linearly in the number of layers and at most logarithmically in the total number of parameters. Our results establish a mathematical basis for regularized estimation of neural networks, and they deepen our mathematical understanding of neural networks and deep learning more generally.
Recommendations
Cites work
- 10.1162/153244303321897690
- Compressed sensing
- Concentration inequalities. A nonasymptotic theory of independence
- Error bounds for approximations with deep ReLU networks
- Estimation and testing under sparsity. École d'Été de Probabilités de Saint-Flour XLV -- 2015
- High-dimensional probability. An introduction with applications in data science
- How Correlations Influence Lasso Prediction
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 49190 (Why is no real title available?)
- scientific article; zbMATH DE number 845714 (Why is no real title available?)
- New concentration inequalities for suprema of empirical processes
- Nonparametric regression using deep neural networks with ReLU activation function
- On the prediction performance of the Lasso
- Oracle inequalities for high-dimensional prediction
- Stable signal recovery from incomplete and inaccurate measurements
- The Bernstein-Orlicz norm and deviation inequalities
- The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network
- Weak convergence and empirical processes. With applications to statistics
Cited in
(13)- Function approximation by deep neural networks with parameters \(\{0, \pm \frac{1}{2}, \pm 1,2\}\)
- On symmetry and initialization for neural networks
- Nonparametric regression with modified ReLU networks
- A statistical model of neural network learning via the Cramer-Rao lower bound
- Regularization theory in the study of generalization ability of a biological neural network model
- Searching for minimal optimal neural networks
- Mean field analysis of neural networks: a law of large numbers
- Guaranteed approximation error estimation of neural networks and model modification
- Statistical guarantees for sparse deep learning
- Layer sparsity in neural networks
- From kernel methods to neural networks: a unifying variational formulation
- Statistical guarantees for distribution estimation of contaminated data via DNN-based MoM-GANs
- Neural drift estimation for ergodic diffusions: nonparametric analysis and numerical exploration
This page was built for publication: Statistical guarantees for regularized neural networks
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6079063)
