ROC curves for regression

DOI10.1016/J.PATCOG.2013.06.014zbMATH Open1326.62138DBLPjournals/pr/Hernandez-Orallo13arXiv1211.2359OpenAlexW1529672735WikidataQ56445167 ScholiaQ56445167MaRDI QIDQ898227FDOQ898227

Authors: J. Hernández-Orallo 

Publication date: 8 December 2015

Published in: Pattern Recognition (Search for Journal in Brave)

Abstract: Several efforts have been done to bring ROC analysis beyond (binary) classification, especially in regression. However, the mapping and possibilities of these proposals do not correspond to what we expect from the analysis of operating conditions, dominance, hybrid methods, etc. In this paper we present a new representation of regression models in the so-called regression ROC (RROC) space. The basic idea is to represent over-estimation on the x axis and under-estimation on the y axis. The curves are just drawn by adjusting a shift, a constant that is added (or subtracted) to the predictions, and plays a similar role as a threshold in classification. From here, we develop the notions of optimal operating condition, convexity, dominance, and explore several evaluation metrics that can be shown graphically, such as the area over the RROC curve (AOC). In particular, we show a novel and significant result, the AOC is equal to the error variance (multiplied by a factor which does not depend on the model). The derivation of RROC curves with non-constant shifts and soft regression models, and the relation with cost plots is also discussed.

Full work available at URL: https://arxiv.org/abs/1211.2359

Recommendations

• Regression Models for Convex ROC Curves
• ROC curves in cost space
• Overestimation of the receiver operating characteristic curve for logistic regression
• A regression modelling framework for receiver operating characteristic curves in medical diagnostic testing
• ROC curves in non-parametric location-scale regression models

zbMATH Keywords

ROC curveserror varianceoperating conditionasymmetric losscost-sensitive regressionMSE decomposition

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) General nonlinear regression (62J02) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Bayesian Estimation and Prediction Using Asymmetric Loss Functions
• Measuring classifier performance: a coherent alternative to the area under the ROC curve
• Measuring the accuracy of diagnostic systems
• A unified view of performance metrics: translating threshold choice into expected classification loss
• Maximizing the area under the ROC curve by pairwise feature combination
• Prediction with a Generalized Cost of Error Function
• Maximizing area under ROC curve for biometric scores fusion
• ROC Curves for Continuous Data
• Bayesian approach to life testing and reliability estimation using asymmetric loss function
• Volume under the ROC surface for multi-class problems.
• ROC curve estimation and hypothesis testing: Applications to breast cancer detection
• ROC curves in cost space
• Partial AUC maximization in a linear combination of dichotomizers
• An online AUC formulation for binary classification
• Iterative Boolean combination of classifiers in the ROC space: an application to anomaly detection with HMMs
• Testing Forecast Optimality Under Unknown Loss
• The ROC manifold for classification systems
• Selecting features in microarray classification using roc curves
• An extension of the Gauss-Newton algorithm for estimation under asymmetric loss

Cited In (11)

• ROC curves in cost space
• R2DS: a novel hierarchical framework for driver fatigue detection in mountain freeway
• Receiver operating characteristic (ROC) movies, universal ROC (UROC) curves, and coefficient of predictive ability (CPA)
• Binarised regression tasks: methods and evaluation metrics
• An Interpretation for the ROC Curve and Inference Using GLM Procedures
• The ROC region of a regression tree
• Selecting cash management models from a multiobjective perspective
• A regression approach to ROC surface, with applications to Alzheimer's disease
• Imbalanced regression and extreme value prediction
• Density-based weighting for imbalanced regression
• Model Checking for ROC Regression Analysis

Uses Software

• R

This page was built for publication: ROC curves for regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q898227)