Cost-Minimising Strategies for Data Labelling: Optimal Stopping and Active Learning

DOI10.1007/978-3-540-77684-0_9MaRDI QIDQ5445296zbMATH OpenOpenAlexFDO

Authors Christos Dimitrakakis, Christian Savu-Krohn

Publication date 4 March 2008

Published in Lecture Notes in Computer Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/0708.1242

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Supervised learning deals with the inference of a distribution over an output or label space $CY$ conditioned on points in an observation space $CX$, given a training dataset $D$ of pairs in $CXimesCY$. However, in a lot of applications of interest, acquisition of large amounts of observations is easy, while the process of generating labels is time-consuming or costly. One way to deal with this problem is {em active} learning, where points to be labelled are selected with the aim of creating a model with better performance than that of an model trained on an equal number of randomly sampled points. In this paper, we instead propose to deal with the labelling cost directly: The learning goal is defined as the minimisation of a cost which is a function of the expected model performance and the total cost of the labels used. This allows the development of general strategies and specific algorithms for (a) optimal stopping, where the expected cost dictates whether label acquisition should continue (b) empirical evaluation, where the cost is used as a performance metric for a given combination of inference, stopping and sampling methods. Though the main focus of the paper is optimal stopping, we also aim to provide the background for further developments and discussion in the related field of active learning.

Recommendations

• Toward optimal probabilistic active learning using a Bayesian approach
• Active learning for cost-sensitive classification
• Optimised probabilistic active learning (OPAL). For fast, non-myopic, cost sensitive active classification
• Surrogate losses in passive and active learning
• Two faces of active learning

Cites work

• scientific article; zbMATH DE number 708500 (Why is no real title available?)
• scientific article; zbMATH DE number 3357742 (Why is no real title available?)
• A decision-theoretic generalization of on-line learning and an application to boosting
• Active Learning in the Non-realizable Case
• Active sampling for class probability estimation and ranking

Cited in

(1)

• Cost-sensitive active learning with a label uniform distribution model

This page was built for publication: Cost-Minimising Strategies for Data Labelling: Optimal Stopping and Active Learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5445296)