Confidence sets with expected sizes for multiclass classification
From MaRDI portal
(Redirected from Publication:4637018)
Nonparametric tolerance and confidence regions (62G15) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Statistical aspects of big data and data science (62R07) Theory of languages and software systems (knowledge-based systems, expert systems, etc.) for artificial intelligence (68T35)
Abstract: Multiclass classification problems such as image annotation can involve a large number of classes. In this context, confusion between classes can occur, and single label classification may be misleading. We provide in the present paper a general device that, given an unlabeled dataset and a score function defined as the minimizer of some empirical and convex risk, outputs a set of class labels, instead of a single one. Interestingly, this procedure does not require that the unlabeled dataset explores the whole classes. Even more, the method is calibrated to control the expected size of the output set while minimizing the classification risk. We show the statistical optimality of the procedure and establish rates of convergence under the Tsybakov margin condition. It turns out that these rates are linear on the number of labels. We apply our methodology to convex aggregation of confidence sets based on the V-fold cross validation principle also known as the superlearning principle. We illustrate the numerical performance of the procedure on real data and demonstrate in particular that with moderate expected size, w.r.t. the number of labels, the procedure provides significant improvement of the classification risk.
Recommendations
- Efficient set-valued prediction in multi-class classification
- Least Ambiguous Set-Valued Classifiers With Bounded Error Levels
- Classification with confidence
- Consistency of plug-in confidence sets for classification in semi-supervised learning
- Minimax semi-supervised set-valued approach to multi-class classification
Cites work
- A decision-theoretic generalization of on-line learning and an application to boosting
- Additive logistic regression: a statistical view of boosting. (With discussion and a rejoinder by the authors)
- Asymptotic Statistics
- Asymptotics of cross-validated risk estimation in estimator selection and performance assess\-ment
- Classification methods with reject option based on convex risk minimization
- Classification with confidence
- Classification with reject option
- Consistency of plug-in confidence sets for classification in semi-supervised learning
- Convexity, Classification, and Risk Bounds
- Distribution-Free Prediction Sets
- Empirical minimization
- scientific article; zbMATH DE number 1332320 (Why is no real title available?)
- scientific article; zbMATH DE number 1966627 (Why is no real title available?)
- scientific article; zbMATH DE number 2168212 (Why is no real title available?)
- Learning nondeterministic classifiers
- On optimum recognition error and reject tradeoff
- On the consistency of multiclass classification methods
- Optimal aggregation of classifiers in statistical learning.
- Probability estimates for multi-class classification by pairwise coupling
- Statistical behavior and consistency of classification methods based on convex risk minimization.
- Super Learner
- Support vector machines with a reject option
Cited in
(12)- Minimax semi-supervised set-valued approach to multi-class classification
- Efficient set-valued prediction in multi-class classification
- KS(conf): a light-weight test if a multiclass classifier operates outside of its specifications
- Optimal rates for nonparametric F-score binary classification via post-processing
- Extrapolating expected accuracies for large multi-class problems
- scientific article; zbMATH DE number 7370598 (Why is no real title available?)
- Consistency of plug-in confidence sets for classification in semi-supervised learning
- Least Ambiguous Set-Valued Classifiers With Bounded Error Levels
- Set-Valued Support Vector Machine with Bounded Error Rates
- Regret-based budgeted decision rules under severe uncertainty
- Prediction intervals with controlled length in the heteroscedastic Gaussian regression
- Robust multi-label classification via preference learning
This page was built for publication: Confidence sets with expected sizes for multiclass classification
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4637018)
