Surrogate regret bounds for generalized classification performance metrics

DOI10.1007/S10994-016-5591-7zbMATH Open1459.62116DBLPjournals/ml/KotlowskiD17arXiv1504.07272OpenAlexW2285335739WikidataQ59609075 ScholiaQ59609075MaRDI QIDQ2398092FDOQ2398092

Wojciech Kotłowski, Krzysztof Dembczyński

Publication date: 15 August 2017

Published in: Machine Learning (Search for Journal in Brave)

Abstract: We consider optimization of generalized performance metrics for binary classification by means of surrogate losses. We focus on a class of metrics, which are linear-fractional functions of the false positive and false negative rates (examples of which include $$-measure, Jaccard similarity coefficient, AM measure, and many others). Our analysis concerns the following two-step procedure. First, a real-valued function $f$ is learned by minimizing a surrogate loss for binary classification on the training sample. It is assumed that the surrogate loss is a strongly proper composite loss function (examples of which include logistic loss, squared-error loss, exponential loss, etc.). Then, given $f$, a threshold $widehatheta$ is tuned on a separate validation sample, by direct optimization of the target performance metric. We show that the regret of the resulting classifier (obtained from thresholding $f$ on $widehatheta$) measured with respect to the target metric is upperbounded by the regret of $f$ measured with respect to the surrogate loss. We also extend our results to cover multilabel classification and provide regret bounds for micro- and macro-averaging measures. Our findings are further analyzed in a computational study on both synthetic and real data sets.

Full work available at URL: https://arxiv.org/abs/1504.07272

Recommendations

• Composite binary losses
• On the Bayes-optimality of F-measure maximizers
• Surrogate regret bounds for bipartite ranking via strongly proper losses
• Calibrated asymmetric surrogate losses
• Calibration and regret bounds for order-preserving surrogate losses in learning to rank

zbMATH Keywords

binary classificationmultilabel classificationregret boundsurrogate loss functionF-measureJaccard similaritygeneralized performance metric

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Introduction to Information Retrieval
• Convexity, Classification, and Risk Bounds
• Title not available (Why is that?)
• Composite binary losses
• On label dependence and loss minimization in multi-label classification
• On the consistency of multi-label learning
• Information, Divergence and Risk for Binary Experiments
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)

Cited In (2)

• Learning with mitigating random consistency from the accuracy measure
• On Loss Functions and Regret Bounds for Multi-Category Classification

Uses Software

• LIBLINEAR
• ElemStatLearn

This page was built for publication: Surrogate regret bounds for generalized classification performance metrics

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2398092)