Consistent algorithms for multiclass classification with an abstain option

DOI10.1214/17-EJS1388zbMATH Open1473.62229arXiv1505.04137OpenAlexW2788508747WikidataQ130196805 ScholiaQ130196805MaRDI QIDQ1697489FDOQ1697489

Authors: Harish G. Ramaswamy, Ambuj Tewari, Shivani Agarwal 

Publication date: 20 February 2018

Published in: Electronic Journal of Statistics (Search for Journal in Brave)

Abstract: We consider the problem of $n$-class classification ($ngeq2$), where the classifier can choose to abstain from making predictions at a given cost, say, a factor $alpha$ of the cost of misclassification. Designing consistent algorithms for such $n$-class classification problems with a `reject option' is the main goal of this paper, thereby extending and generalizing previously known results for $n=2$. We show that the Crammer-Singer surrogate and the one vs all hinge loss, albeit with a different predictor than the standard argmax, yield consistent algorithms for this problem when $alpha=frac12$. More interestingly, we design a new convex surrogate that is also consistent for this problem when $alpha=frac12$ and operates on a much lower dimensional space ($log(n)$ as opposed to $n$). We also generalize all three surrogates to be consistent for any $alphain[0,frac12]$.

Full work available at URL: https://arxiv.org/abs/1505.04137

Recommendations

• Classification methods with reject option based on convex risk minimization
• Classification with a reject option using a hinge loss
• Classification with reject option
• On reject and refine options in multicategory classification
• Multilabel classification with partial abstention: Bayes-optimal prediction under label independence

zbMATH Keywords

classificationmachine learningcalibrationsurrogate lossstatistical consistencyabstain loss

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Pattern recognition, speech recognition (68T10)

Cites Work

• In defense of one-vs-all classification
• Title not available (Why is that?)
• Multicategory Support Vector Machines
• Reducing multiclass to binary: A unifying approach for margin classifiers
• 10.1162/15324430260185628
• On the consistency of multiclass classification methods
• On optimum recognition error and reject tradeoff
• Classification with a reject option using a hinge loss
• Statistical analysis of some multi-category large margin classification methods
• Title not available (Why is that?)
• Classification methods with reject option based on convex risk minimization
• Title not available (Why is that?)
• Classification using geometric level sets
• On the foundations of noise-free selective classification
• Learning with Rejection
• Analysis of error-reject trade-off in linearly combined multiple classifiers
• Composite multiclass losses
• Title not available (Why is that?)
• On reject and refine options in multicategory classification
• Structural, Syntactic, and Statistical Pattern Recognition

Cited In (7)

• Optimal rates for nonparametric F-score binary classification via post-processing
• Multilabel classification with partial abstention: Bayes-optimal prediction under label independence
• Title not available (Why is that?)
• Efficient set-valued prediction in multi-class classification
• Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
• Least Ambiguous Set-Valued Classifiers With Bounded Error Levels
• Multi-stage classifier design

Uses Software

• SVMlight

This page was built for publication: Consistent algorithms for multiclass classification with an abstain option

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1697489)