Efficient set-valued prediction in multi-class classification
From MaRDI portal
Abstract: In cases of uncertainty, a multi-class classifier preferably returns a set of candidate classes instead of predicting a single class label with little guarantee. More precisely, the classifier should strive for an optimal balance between the correctness (the true class is among the candidates) and the precision (the candidates are not too many) of its prediction. We formalize this problem within a general decision-theoretic framework that unifies most of the existing work in this area. In this framework, uncertainty is quantified in terms of conditional class probabilities, and the quality of a predicted set is measured in terms of a utility function. We then address the problem of finding the Bayes-optimal prediction, i.e., the subset of class labels with highest expected utility. For this problem, which is computationally challenging as there are exponentially (in the number of classes) many predictions to choose from, we propose efficient algorithms that can be applied to a broad family of utility functions. Our theoretical results are complemented by experimental studies, in which we analyze the proposed algorithms in terms of predictive accuracy and runtime efficiency.
Recommendations
- Multilabel prediction with probability sets: the Hamming loss case
- Multilabel predictions with sets of probabilities: the Hamming and ranking loss cases
- Least Ambiguous Set-Valued Classifiers With Bounded Error Levels
- Distribution-free, Risk-controlling Prediction Sets
- Confidence sets with expected sizes for multiclass classification
Cites work
- scientific article; zbMATH DE number 2168212 (Why is no real title available?)
- scientific article; zbMATH DE number 6306019 (Why is no real title available?)
- A family of admissible heuristics for \(\mathrm{A}^*\) to perform inference in probabilistic classifier chains
- A tutorial on conformal prediction
- An analysis of chaining in multi-label classification
- Confidence sets with expected sizes for multiclass classification
- Consistent algorithms for multiclass classification with an abstain option
- Evaluating credal classifiers by utility-discounted predictive accuracy
- Exact and efficient top-\(K\) inference for multi-target prediction by querying separable linear relational models
- LIBLINEAR: a library for large linear classification
- Learning nondeterministic classifiers
- Learning reliable classifiers from small or incomplete data sets: the naive credal classifier 2
- On the Bayes-optimality of F-measure maximizers
- On the effectiveness of heuristics for learning nested dichotomies: an empirical analysis
- Reliable classification: learning classifiers that distinguish aleatoric and epistemic uncertainty
Cited in
(9)- Confidence sets with expected sizes for multiclass classification
- Multilabel predictions with sets of probabilities: the Hamming and ranking loss cases
- Cautious decision-making for tree ensembles
- Learning sets of probabilities through ensemble methods
- Distribution-free, Risk-controlling Prediction Sets
- Extrapolating expected accuracies for large multi-class problems
- Least Ambiguous Set-Valued Classifiers With Bounded Error Levels
- Multilabel prediction with probability sets: the Hamming loss case
- Learning nondeterministic classifiers
Describes a project that uses
Uses Software
This page was built for publication: Efficient set-valued prediction in multi-class classification
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2036773)
