Convergence rates of deep ReLU networks for multiclass classification
From MaRDI portal
Abstract: For classification problems, trained deep neural networks return probabilities of class memberships. In this work we study convergence of the learned probabilities to the true conditional class probabilities. More specifically we consider sparse deep ReLU network reconstructions minimizing cross-entropy loss in the multiclass classification setup. Interesting phenomena occur when the class membership probabilities are close to zero. Convergence rates are derived that depend on the near-zero behaviour via a margin-type condition.
Recommendations
- Fast convergence rates of deep neural networks for classification
- Error bounds for approximations with deep ReLU networks
- On the rate of convergence of fully connected deep neural network regression estimates
- Approximation of smoothness classes by deep rectifier networks
- Nonlinear approximation and (deep) ReLU networks
Cites work
- scientific article; zbMATH DE number 1739768 (Why is no real title available?)
- scientific article; zbMATH DE number 1420699 (Why is no real title available?)
- A moment bound for multi-hinge classifiers
- Convexity, Classification, and Risk Bounds
- Error bounds for approximations with deep ReLU networks
- Fast convergence rates of deep neural networks for classification
- Fast learning rates for plug-in classifiers
- Inequalities for quasiconformal mappings in space
- Information-theoretic determination of minimax rates of convergence
- Minimum contrast estimators on sieves: Exponential bounds and rates of convergence
- Mutual information, metric entropy and cumulative relative entropy risk
- Nonparametric Regression Based on Hierarchical Interaction Models
- Nonparametric regression using deep neural networks with ReLU activation function
- On deep learning as a remedy for the curse of dimensionality in nonparametric regression
- On maximum likelihood estimation in infinite dimensional parameter spaces
- On the rate of convergence of fully connected deep neural network regression estimates
- Optimal aggregation of classifiers in statistical learning.
- Probability Inequalities for the Sum of Independent Random Variables
- Probability inequalities for likelihood ratios and convergence rates of sieve MLEs
- Rate-optimal estimation for a general class of nonparametric regression models with unknown link functions
- Rényi Divergence and Kullback-Leibler Divergence
- Smooth discrimination analysis
- Weak convergence and empirical processes. With applications to statistics
Cited in
(12)- Analysis of convolutional neural network image classifiers in a hierarchical max-pooling model with additional local pooling
- Adaptive novelty detection with false discovery rate guarantee
- On the convergence of formally diverging neural net-based classifiers
- Convergence of deep convolutional neural networks
- Statistical theory for image classification using deep convolutional neural network with cross-entropy loss under the hierarchical max-pooling model
- Learning Theory
- Optimal convergence rates of deep neural networks in a classification setting
- Multiclass classification for multidimensional functional data through deep neural networks
- Functional data analysis using deep neural networks
- Fast convergence rates of deep neural networks for classification
- Globally Convergent Multilevel Training of Deep Residual Networks
- Monotone learning with rectified wire networks
This page was built for publication: Convergence rates of deep ReLU networks for multiclass classification
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2137813)
