Contrasting probabilistic scoring rules

DOI10.1016/J.JSPI.2013.05.012zbMATH Open1279.62111arXiv1112.4530OpenAlexW2008940396MaRDI QIDQ394786FDOQ394786

Authors: R. L. Machete 

Publication date: 27 January 2014

Published in: Journal of Statistical Planning and Inference (Search for Journal in Brave)

Abstract: There are several scoring rules that one can choose from in order to score probabilistic forecasting models or estimate model parameters. Whilst it is generally agreed that proper scoring rules are preferable, there is no clear criterion for preferring one proper scoring rule above another. This manuscript compares and contrasts some commonly used proper scoring rules and provides guidance on scoring rule selection. In particular, it is shown that the logarithmic scoring rule prefers erring with more uncertainty, the spherical scoring rule prefers erring with lower uncertainty, whereas the other scoring rules are indifferent to either option.

Full work available at URL: https://arxiv.org/abs/1112.4530

Recommendations

• Theory and applications of proper scoring rules
• Strictly Proper Scoring Rules, Prediction, and Estimation
• Choosing a strictly proper scoring rule
• Should Scoring Rules be “Effective”?
• Additive scoring rules for discrete sample spaces

zbMATH Keywords

estimationutility functionsforecast evaluationsprobabilistic forecasting

Mathematics Subject Classification ID

Inference from stochastic processes and prediction (62M20) Nonparametric inference (62G99)

Cites Work

• Strictly Proper Scoring Rules, Prediction, and Estimation
• Probabilistic Forecasts, Calibration and Sharpness
• A Mathematical Theory of Communication
• Probability Theory
• Elicitation of Personal Probabilities and Expectations
• Prospect Theory: An Analysis of Decision under Risk
• Should Scoring Rules be “Effective”?
• Title not available (Why is that?)
• Scoring Rules, Generalized Entropy, and Utility Maximization
• Effective Scoring Rules for Probabilistic Forecasts
• Initial distribution spread: a density forecasting approach
• Early warning with calibrated and sharper probabilistic forecasts
• Tailored scoring rules for probabilities
• Fitting probability forecasting models by scoring rules and maximum likelihood

Cited In (22)

• Scoring, truthlikeness, and value
• Borda's paradox with weighted scoring rules
• Scoring rules for statistical models on spheres
• Choosing a strictly proper scoring rule
• Scoring rules and the evaluation of probabilities. (With discussion)
• Linear scoring rules for probabilistic binary classification
• Score-Based vs. Probability-Based Enumeration – A Cautionary Note
• Minimum scoring rule inference
• Asymptotic minimum scoring rule prediction
• A characterization for the spherical scoring rule
• Lumpy and intermittent retail demand forecasts with score-driven models
• Objective Bayesian inference with proper scoring rules
• Evaluating Probabilities: Asymmetric Scoring Rules
• Higher-order asymptotics for scoring rules
• A Probability Scoring Rule for Simultaneous Events
• Consistency of scoring rules: a reinvestigation of composition-consistency
• Comment on article by Dawid and Musio
• Semi-supervised nonparametric Bayesian modelling of spatial proteomics
• Scoring rules over subsets of alternatives: consistency and paradoxes
• Robust confidence distributions from proper scoring rules
• A construction principle for proper scoring rules
• An Overview of Applications of Proper Scoring Rules

This page was built for publication: Contrasting probabilistic scoring rules

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q394786)