Performance measures for classification systems with rejection

DOI10.1016/J.PATCOG.2016.10.011MaRDI QIDQ2290329zbMATH OpenOpenAlexFDO

Authors Filipe Condessa, Jelena Kovačević, José M. Bioucas-Dias

Publication date 27 January 2020

Published in Pattern Recognition (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1504.02763

zbMATH Keywords

classification with rejectionperformance measures

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Pattern recognition, speech recognition (68T10)

Abstract: Classifiers with rejection are essential in real-world applications where misclassifications and their effects are critical. However, if no problem specific cost function is defined, there are no established measures to assess the performance of such classifiers. We introduce a set of desired properties for performance measures for classifiers with rejection, based on which we propose a set of three performance measures for the evaluation of the performance of classifiers with rejection that satisfy the desired properties. The nonrejected accuracy measures the ability of the classifier to accurately classify nonrejected samples; the classification quality measures the correct decision making of the classifier with rejector; and the rejection quality measures the ability to concentrate all misclassified samples onto the set of rejected samples. From the measures, we derive the concept of relative optimality that allows us to connect the measures to a family of cost functions that take into account the trade-off between rejection and misclassification. We illustrate the use of the proposed performance measures on classifiers with rejection applied to synthetic and real-world data.

Recommendations

• scientific article; zbMATH DE number 1759566
• Learning with Rejection
• A family of measures for best top-\(n\) class-selective decision rules
• Classification with reject option
• ANALYSIS OF CLASSIFICATION WITH A REJECT OPTION

Cites work

• A \(k\)-order fuzzy OR operator for pattern classification with \(k\)-order ambiguity rejection
• A family of measures for best top-\(n\) class-selective decision rules
• A unified view of class-selection with probabilistic classifiers
• Adaptive Relevance Matrices in Learning Vector Quantization
• Analysis of error-reject trade-off in linearly combined multiple classifiers
• Classification methods with reject option based on convex risk minimization
• Classification with a reject option using a hinge loss
• Classification with reject option
• scientific article; zbMATH DE number 1759566 (Why is no real title available?)
• scientific article; zbMATH DE number 2089926 (Why is no real title available?)
• Image classification for content-based indexing
• Lasso type classifiers with a reject option
• On optimum recognition error and reject tradeoff
• On the use of ROC analysis for the optimization of abstaining classifiers
• Reject strategies driven combination of pattern classifiers
• Structural, Syntactic, and Statistical Pattern Recognition
• Support vector machines with a reject option

Cited in

(6)

• scientific article; zbMATH DE number 2125361 (Why is no real title available?)
• scientific article; zbMATH DE number 1759566 (Why is no real title available?)
• A family of measures for best top-\(n\) class-selective decision rules
• Explainability through uncertainty: trustworthy decision-making with neural networks
• Credal ensembling in multi-class classification
• Predictable artificial intelligence

This page was built for publication: Performance measures for classification systems with rejection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2290329)