Optimal convergence rates of deep neural networks in a classification setting

DOI10.1214/23-EJS2187MaRDI QIDQ6184926zbMATH OpenFDO

Authors Joseph T. Meyer

Publication date 5 January 2024

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2207.12180, https://projecteuclid.org/journals/electronic-journal-of-statistics/volume-17/issue-2/Optimal-convergence-rates-of-deep-neural-networks-in-a-classification/10.1214/23-EJS2187.full

zbMATH Keywords

classificationdeep neural networksTsybakov noise condition

Mathematics Subject Classification ID

Minimax procedures in statistical decision theory (62C20) Nonparametric estimation (62G05)

Abstract: We establish optimal convergence rates up to a log-factor for a class of deep neural networks in a classification setting under a restraint sometimes referred to as the Tsybakov noise condition. We construct classifiers in a general setting where the boundary of the bayes-rule can be approximated well by neural networks. Corresponding rates of convergence are proven with respect to the misclassification error. It is then shown that these rates are optimal in the minimax sense if the boundary satisfies a smoothness condition. Non-optimal convergence rates already exist for this setting. Our main contribution lies in improving existing rates and showing optimality, which was an open problem. Furthermore, we show almost optimal rates under some additional restraints which circumvent the curse of dimensionality. For our analysis we require a condition which gives new insight on the restraint used. In a sense it acts as a requirement for the "correct noise exponent" for a class of functions.

Cites work

• Approximation and estimation bounds for artificial neural networks
• Approximation by superpositions of a sigmoidal function
• Classifiers of support vector machine type with \(\ell_1\) complexity regularization
• Convergence rates of deep ReLU networks for multiclass classification
• Error bounds for approximations with deep ReLU networks
• Estimation of a Function of Low Local Dimensionality by Deep Neural Networks
• Fast convergence rates of deep neural networks for classification
• Fast learning rates for plug-in classifiers
• Metric entropy of some classes of sets with differentiable boundaries
• Nonparametric regression using deep neural networks with ReLU activation function
• On the rate of convergence of fully connected deep neural network regression estimates
• Optimal aggregation of classifiers in statistical learning.
• Optimal approximation of piecewise smooth functions using deep ReLU neural networks
• Smooth discrimination analysis
• SVM Soft Margin Classifiers: Linear Programming versus Quadratic Programming

Cited in

(4)

• On convergence rate of projection neural networks
• Learning Theory
• Misclassification bounds for PAC-Bayesian sparse deep learning
• Statistical learnability of smooth boundaries via pairwise binary classification with deep ReLU networks

This page was built for publication: Optimal convergence rates of deep neural networks in a classification setting

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6184926)