Convergence rates of deep ReLU networks for multiclass classification
From MaRDI portal
Abstract: For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true conditional class probabilities. More specifically we consider sparse deep ReLU network reconstructions minimizing cross-entropy loss in the multiclass classification setup. Interesting phenomena occur when the class membership probabilities are close to zero. Convergence rates are derived that depend on the near-zero behaviour via a margin-type condition.
Recommendations
- Fast convergence rates of deep neural networks for classification
- Error bounds for approximations with deep ReLU networks
- On the rate of convergence of fully connected deep neural network regression estimates
- Approximation of smoothness classes by deep rectifier networks
- Nonlinear approximation and (deep) ReLU networks
Cites work
- A moment bound for multi-hinge classifiers
- Convexity, Classification, and Risk Bounds
- Error bounds for approximations with deep ReLU networks
- Fast convergence rates of deep neural networks for classification
- Fast learning rates for plug-in classifiers
- scientific article; zbMATH DE number 1739768 (Why is no real title available?)
- scientific article; zbMATH DE number 1420699 (Why is no real title available?)
- Inequalities for quasiconformal mappings in space
- Information-theoretic determination of minimax rates of convergence
- Minimum contrast estimators on sieves: Exponential bounds and rates of convergence
- Mutual information, metric entropy and cumulative relative entropy risk
- Nonparametric Regression Based on Hierarchical Interaction Models
- Nonparametric regression using deep neural networks with ReLU activation function
- On deep learning as a remedy for the curse of dimensionality in nonparametric regression
- On maximum likelihood estimation in infinite dimensional parameter spaces
- On the rate of convergence of fully connected deep neural network regression estimates
- Optimal aggregation of classifiers in statistical learning.
- Probability inequalities for likelihood ratios and convergence rates of sieve MLEs
- Probability Inequalities for the Sum of Independent Random Variables
- Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
- Rényi Divergence and Kullback-Leibler Divergence
- Smooth discrimination analysis
- Weak convergence and empirical processes. With applications to statistics
Cited in
(20)- On the convergence of formally diverging neural net-based classifiers
- Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local pooling
- Monotone learning with rectified wire networks
- Traces of class/cross-class structure pervade deep learning spectra
- Learning Theory
- Convergence of deep convolutional neural networks
- Fast convergence rates of deep neural networks for classification
- Globally Convergent Multilevel Training of Deep Residual Networks
- Adaptive novelty detection with false discovery rate guarantee
- Optimal convergence rates of deep neural networks in a classification setting
- Multiclass classification for multidimensional functional data through deep neural networks
- Statistical theory for image classification using deep convolutional neural network with cross-entropy loss under the hierarchical max-pooling model
- Functional data analysis using deep neural networks
- On the rates of convergence for learning with convolutional neural networks
- Misclassification bounds for PAC-Bayesian sparse deep learning
- Statistical inference of optimal allocations. I: Regularities and their implications
- Generative semi-supervised classification
- Statistical learnability of smooth boundaries via pairwise binary classification with deep ReLU networks
- A primer on Bayesian neural networks: review and debates
- A novel statistical approach to analyze image classification
This page was built for publication: Convergence rates of deep ReLU networks for multiclass classification
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2137813)
