“Calibeating”: Beating forecasters at their own game
From MaRDI portal
Publication:6180412
Abstract: In order to identify expertise, forecasters should not be tested by their calibration score, which can always be made arbitrarily small, but rather by their Brier score. The Brier score is the sum of the calibration score and the refinement score; the latter measures how good the sorting into bins with the same forecast is, and thus attests to "expertise." This raises the question of whether one can gain calibration without losing expertise, which we refer to as "calibeating." We provide an easy way to calibeat any forecast, by a deterministic online procedure. We moreover show that calibeating can be achieved by a stochastic procedure that is itself calibrated, and then extend the results to simultaneously calibeating multiple procedures, and to deterministic procedures that are continuously calibrated.
Cites work
- A proof of calibration via Blackwell's approachability theorem.
- An analog of the minimax theorem for vector payoffs
- Asymptotic calibration
- Competitive On-line Statistics
- scientific article; zbMATH DE number 1099349 (Why is no real title available?)
- scientific article; zbMATH DE number 1390070 (Why is no real title available?)
- scientific article; zbMATH DE number 3078991 (Why is no real title available?)
- Prediction in the worst case
- Prediction, Learning, and Games
- Probability theory
- Relative loss bounds for on-line density estimation with the exponential family of distributions
- Self-Calibrating Priors Do Not Exist
- Smooth calibration, leaky forecasts, finite recall, and Nash dynamics
- The Well-Calibrated Bayesian
- Univariate Discrete Distributions
This page was built for publication: “Calibeating”: Beating forecasters at their own game
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6180412)
