Generalization Error in Deep Learning
From MaRDI portal
Publication:3296180
Abstract: Deep learning models have lately shown great performance in various fields such as computer vision, speech recognition, speech translation, and natural language processing. However, alongside their state-of-the-art performance, it is still generally unclear what is the source of their generalization ability. Thus, an important question is what makes deep neural networks able to generalize well from the training set to new data. In this article, we provide an overview of the existing theory and bounds for the characterization of the generalization error of deep neural networks, combining both classical and more recent theoretical and empirical results.
Recommendations
- Generalization in Deep Learning
- scientific article; zbMATH DE number 7387621
- An analysis of training and generalization errors in shallow and deep networks
- Generalization errors of the simple perceptron
- scientific article; zbMATH DE number 1453028
- scientific article; zbMATH DE number 1728675
- Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness
- Generalization Error Analysis of Neural Networks with Gradient Based Regularization
- Generalization in Overparameterized Models
Cites work
- 10.1162/153244302760200704
- 10.1162/153244303321897690
- Convergence radius and sample complexity of ITKM algorithms for dictionary learning
- Convolutional neural networks analyzed via convolutional sparse coding
- Deep learning
- Elements of Information Theory
- Machine learning. A probabilistic perspective
- Neural Network Learning
- PMTK
- Robust Large Margin Deep Neural Networks
- Robustness and generalization
- Sample Complexity of Dictionary Learning and Other Matrix Factorizations
- Simplified PAC-Bayesian margin bounds.
- Some PAC-Bayesian theorems
- Sparse and Spurious: Dictionary Learning With Noise and Outliers
- The implicit bias of gradient descent on separable data
- The sample complexity of dictionary learning
- Understanding machine learning. From theory to algorithms
- Weak convergence and empirical processes. With applications to statistics
Cited in
(12)- Over-parametrized deep neural networks minimizing the empirical risk do not generalize well
- Generalization in Deep Learning
- Inference for the generalization error
- Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation
- An analysis of training and generalization errors in shallow and deep networks
- High-dimensional dynamics of generalization error in neural networks
- Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness
- Generality's price: Inescapable deficiencies in machine-learned programs
- Generalization error of GAN from the discriminator's perspective
- scientific article; zbMATH DE number 7064043 (Why is no real title available?)
- Optimization for deep learning: an overview
- High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
This page was built for publication: Generalization Error in Deep Learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3296180)
