Generalization Error in Deep Learning
From MaRDI portal
Publication:3296180
DOI10.1007/978-3-319-73074-5_5zbMATH Open1494.68240arXiv1808.01174OpenAlexW2887344700MaRDI QIDQ3296180FDOQ3296180
Authors: Daniel Jakubovitz, M. R. D. Rodrigues, Raja Giryes 
Publication date: 7 July 2020
Published in: Applied and Numerical Harmonic Analysis (Search for Journal in Brave)
Abstract: Deep learning models have lately shown great performance in various fields such as computer vision, speech recognition, speech translation, and natural language processing. However, alongside their state-of-the-art performance, it is still generally unclear what is the source of their generalization ability. Thus, an important question is what makes deep neural networks able to generalize well from the training set to new data. In this article, we provide an overview of the existing theory and bounds for the characterization of the generalization error of deep neural networks, combining both classical and more recent theoretical and empirical results.
Full work available at URL: https://arxiv.org/abs/1808.01174
Recommendations
- Generalization in Deep Learning
- scientific article; zbMATH DE number 7387621
- An analysis of training and generalization errors in shallow and deep networks
- Generalization errors of the simple perceptron
- Publication:4955300
- scientific article; zbMATH DE number 1728675
- Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness
- Generalization Error Analysis of Neural Networks with Gradient Based Regularization
- Generalization in Overparameterized Models
Cites Work
- PMTK
- Weak convergence and empirical processes. With applications to statistics
- Elements of Information Theory
- Machine learning. A probabilistic perspective
- 10.1162/153244302760200704
- Deep learning
- Understanding machine learning. From theory to algorithms
- Convolutional neural networks analyzed via convolutional sparse coding
- 10.1162/153244303321897690
- Neural Network Learning
- The sample complexity of dictionary learning
- Simplified PAC-Bayesian margin bounds.
- Robustness and generalization
- Some PAC-Bayesian theorems
- Sample Complexity of Dictionary Learning and Other Matrix Factorizations
- Convergence radius and sample complexity of ITKM algorithms for dictionary learning
- Sparse and Spurious: Dictionary Learning With Noise and Outliers
- Robust Large Margin Deep Neural Networks
- The implicit bias of gradient descent on separable data
Cited In (12)
- Over-parametrized deep neural networks minimizing the empirical risk do not generalize well
- Generalization in Deep Learning
- Inference for the generalization error
- Out-of-distributional risk bounds for neural operators with applications to the Helmholtz equation
- An analysis of training and generalization errors in shallow and deep networks
- High-dimensional dynamics of generalization error in neural networks
- Quantifying the generalization error in deep learning in terms of data distribution and neural network smoothness
- Generality's price: Inescapable deficiencies in machine-learned programs
- Generalization error of GAN from the discriminator's perspective
- Title not available (Why is that?)
- High-Dimensional Learning Under Approximate Sparsity with Applications to Nonsmooth Estimation and Regularized Neural Networks
- Optimization for deep learning: an overview
Uses Software
This page was built for publication: Generalization Error in Deep Learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3296180)
