Risk bounds for statistical learning
From MaRDI portal
Classificationconcentration inequalitiesempirical processesentropy with bracketingminimax estimationmodel selectionpattern recognitionregression estimationstructural minimization of riskVC-classVC-dimension
Inequalities; stochastic orderings (60E15) Statistical aspects of information-theoretic topics (62B10) Bayesian inference (62F15) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05) Measures of information, entropy (94A17)
Abstract: We propose a general theorem providing upper bounds for the risk of an empirical risk minimizer (ERM).We essentially focus on the binary classification framework. We extend Tsybakov's analysis of the risk of an ERM under margin type conditions by using concentration inequalities for conveniently weighted empirical processes. This allows us to deal with ways of measuring the ``size of a class of classifiers other than entropy with bracketing as in Tsybakov's work. In particular, we derive new risk bounds for the ERM when the classification rules belong to some VC-class under margin conditions and discuss the optimality of these bounds in a minimax sense.
Recommendations
- Lower bounds on the bayes risk eor statistical prediction problems
- Risk bounds of learning processes for Lévy processes
- A remark about a learning risk lower bound
- Excess risk bounds in robust empirical risk minimization
- On Bayes risk lower bounds
- Risk bounds when learning infinitely many response functions by ordinary linear regression
- scientific article; zbMATH DE number 4056794
- Risk bounds for random regression graphs
- Finite-Sample Risk Bounds for Maximum Likelihood Estimation With Arbitrary Penalties
Cites work
- A Bennett concentration inequality and its application to suprema of empirical processes
- A New Lower Bound for Multiple Hypothesis Testing
- Adaptive estimation of the intensity of inhomogeneous Poisson processes via concentration inequalities
- Estimation of dependences based on empirical data. Transl. from the Russian by Samuel Kotz
- scientific article; zbMATH DE number 4170917 (Why is no real title available?)
- scientific article; zbMATH DE number 4032498 (Why is no real title available?)
- scientific article; zbMATH DE number 49190 (Why is no real title available?)
- scientific article; zbMATH DE number 1301684 (Why is no real title available?)
- scientific article; zbMATH DE number 1064667 (Why is no real title available?)
- scientific article; zbMATH DE number 3795074 (Why is no real title available?)
- scientific article; zbMATH DE number 3795075 (Why is no real title available?)
- scientific article; zbMATH DE number 3446442 (Why is no real title available?)
- Information-theoretic determination of minimax rates of convergence
- Minimax theory of image reconstruction
- Minimum contrast estimators on sieves: Exponential bounds and rates of convergence
- New concentration inequalities in product spaces
- Optimal aggregation of classifiers in statistical learning.
- Predicting \(\{ 0,1\}\)-functions on randomly drawn points
- Risk bounds for model selection via penalization
- Smooth discrimination analysis
- Some applications of concentration inequalities to statistics
- Sphere packing numbers for subsets of the Boolean \(n\)-cube with bounded Vapnik-Chervonenkis dimension
- Uniform Central Limit Theorems
Cited in
(only showing first 100 items - show all)- Bounds on margin distributions in learning problems
- Learning from binary labels with instance-dependent noise
- Best subset binary prediction
- On concentration for (regularized) empirical risk minimization
- Discussion of ``On concentration for (regularized) empirical risk minimization by Sara van de Geer and Martin Wainwright
- Relative deviation learning bounds and generalization with unbounded loss functions
- Improved classification rates under refined margin conditions
- Optimal upper and lower bounds for the true and empirical excess risks in heteroscedastic least-squares regression
- Upper bounds and aggregation in bipartite ranking
- Classification with minimax fast rates for classes of Bayes rules with sparse representation
- Penalized empirical risk minimization over Besov spaces
- A local maximal inequality under uniform entropy
- Minimax semi-supervised set-valued approach to multi-class classification
- Model selection in utility-maximizing binary prediction
- A MOM-based ensemble method for robustness, subsampling and hyperparameter tuning
- Set structured global empirical risk minimizers are rate optimal in general dimensions
- Optimal linear discriminators for the discrete choice model in growing dimensions
- A no-free-lunch theorem for multitask learning
- On least squares estimation under heteroscedastic and heavy-tailed errors
- On robust learning in the canonical change point problem under heavy tailed errors in finite and growing dimensions
- Optimal functional supervised classification with separation condition
- Robust machine learning by median-of-means: theory and practice
- On biased random walks, corrupted intervals, and learning under adversarial design
- Learning the distribution of latent variables in paired comparison models with round-robin scheduling
- Convergence rates of least squares regression estimators with heavy-tailed errors
- Exact lower bounds for the agnostic probably-approximately-correct (PAC) machine learning model
- Nonasymptotic bounds for vector quantization in Hilbert spaces
- Minimax fast rates for discriminant analysis with errors in variables
- Local Rademacher complexities and oracle inequalities in risk minimization. (2004 IMS Medallion Lecture). (With discussions and rejoinder)
- Model selection by bootstrap penalization for classification
- Statistical performance of support vector machines
- Ranking and empirical minimization of \(U\)-statistics
- Rates of convergence in active learning
- Empirical risk minimization is optimal for the convex aggregation problem
- Simultaneous adaptation to the margin and to complexity in classification
- Optimal rates of aggregation in classification under low noise assumption
- Distinctive features of minimization of a risk functional in mass data sets
- Concentration inequalities and asymptotic results for ratio type empirical processes
- Bandwidth selection in kernel empirical risk minimization via the gradient
- Gibbs posterior concentration rates under sub-exponential type losses
- Learning without concentration
- Measuring the capacity of sets of functions in the analysis of ERM
- Risk minimization and minimum description for linear discriminant functions
- Fast rate of convergence in high-dimensional linear discriminant analysis
- On the optimality of sample-based estimates of the expectation of the empirical minimizer
- Asymptotics in empirical risk minimization
- Theory of Classification: a Survey of Some Recent Advances
- scientific article; zbMATH DE number 5158916
- Noisy discriminant analysis with boundary assumptions
- Fast rates for empirical vector quantization
- Improved Risk Tail Bounds for On-Line Algorithms
- Inverse statistical learning
- A statistical view of clustering performance through the theory of U-processes
- Risk bounds for CART classifiers under a margin condition
- General nonexact oracle inequalities for classes with a subexponential envelope
- Margin-adaptive model selection in statistical learning
- A theory of learning with corrupted labels
- Theoretical analysis of cross-validation for estimating the risk of the \(k\)-nearest neighbor classifier
- Learning Theory
- Classification algorithms using adaptive partitioning
- Optimal exponential bounds on the accuracy of classification
- Fast rates for general unbounded loss functions: from ERM to generalized Bayes
- scientific article; zbMATH DE number 7370646 (Why is no real title available?)
- A new method for estimation and model selection: \(\rho\)-estimation
- Sample complexity of sample average approximation for conditional stochastic optimization
- scientific article; zbMATH DE number 7306919 (Why is no real title available?)
- A contextual bandit bake-off
- Classification with reject option
- Random subclass bounds.
- Active learning for cost-sensitive classification
- scientific article; zbMATH DE number 7064063 (Why is no real title available?)
- Risk Bounds for CART Regression Trees
- Risk bounds for new M-estimation problems
- Multiclass learnability and the ERM principle
- Learning with risks based on M-location
- Sparse quantile regression
- Re-thinking high-dimensional mathematical statistics. Abstracts from the workshop held May 15--21, 2022
- Diametrical risk minimization: theory and computations
- Orthogonal statistical learning
- PAC learning halfspaces in non-interactive local differential privacy model with public unlabeled data
- Nonexact oracle inequalities, r-learnability, and fast rates
- Robust supervised learning with coordinate gradient descent
- Sharper lower bounds on the performance of the empirical risk minimization algorithm
- A strongly polynomial algorithm for approximate Forster transforms and its application to halfspace learning
- Optimal model selection for density estimation of stationary data under various mixing condi\-tions
- On regression and classification with possibly missing response variables in the data
- A notion of uniqueness for the adversarial Bayes classifier
- Minimax rates of convergence for the nonparametric estimation of the diffusion coefficient from time-homogeneous SDE paths
- On nonparametric regression with incomplete data via inverse-weighting and their convergence in L_p norms
- Binary regression and classification with covariates in metric spaces
- ALM-PU: positive and unlabeled learning with constrained optimization
- Label ranking through nonparametric regression
- Dueling optimization with a monotone adversary
- Iterative regularization in classification via hinge loss diagonal descent
- Attribute-efficient learning of halfspaces with malicious noise: near-optimal label complexity and noise tolerance
- A tight excess risk bound via a unified PAC-Bayesian-Rademacher-Shtarkov-MDL complexity
- A high-dimensional Wilks phenomenon
- Statistical learnability of smooth boundaries via pairwise binary classification with deep ReLU networks
- A novel statistical approach to analyze image classification
- Fast learning rates for plug-in classifiers
This page was built for publication: Risk bounds for statistical learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q869973)
