Proper scoring rules and Bregman divergence

DOI10.3150/16-BEJ857MaRDI QIDQ2405175zbMATH OpenFDO

Authors Evgeni Y. Ovcharov

Publication date 21 September 2017

Published in Bernoulli (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1502.01178, https://projecteuclid.org/euclid.bj/1501142436

zbMATH Keywords

Bregman divergencecharacterisationconvex analysisentropyproper scoring rulerobust estimationsubgradient

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Nonparametric robustness (62G35) Nonparametric inference (62G99)

Abstract: We revisit the mathematical foundations of proper scoring rules (PSRs) and Bregman divergences and present their characteristic properties in a unified theoretical framework. In many situations it is preferable not to generate a PSR directly from its convex entropy on the unit simplex but instead by the sublinear extension of the entropy to the positive orthant. This gives the scoring rule simply as a subgradient of the extended entropy, allowing for a more elegant theory. The other convex extensions of the entropy generate affine extensions of the scoring rule and induce the class of functional Bregman divergences. We discuss the geometric nature of the relationship between PSRs and Bregman divergences and extend and unify existing partial results. We also approach the topic of differentiability of entropy functions. Not all entropies of interest possess functional derivatives, but they do all have directional derivatives in almost every direction. Relying on the notion of quasi-interior of a convex set to quantify the latter property, we formalise under what conditions a PSR may be considered to be uniquely determined from its entropy.

Review text

From the author's introduction: ``Proper scoring rules measure the quality of probabilistic forecasts. They can be used as protocols for eliciting private information and as incentives for a forecaster to make truthful predictions. Bregman divergences, on the other hand, originate in convex programming as generalizations of Euclidean metric. The two notions are intimately connected through the fact that they both derive from convex functions known as entropies. The principal objective of the present work is to present and and review the mathematical properties of the two entities in a unified theoretical framework. To that end, we systematically generalize a number of results from the Euclidean setting to the framework of spaces of finite signed measures. In this setting score and Bregman divergences may be identified. The author also formalizes the proper affine scoring rules motivating this by proper estimation problems. Finally, he develops the elements of the regularity theory of entropy functions. This allows him to describe under what conditions a general convex function can be identified with the entropy function of a proper scoring rule and whether this association is unique.

Recommendations

• Affine invariant divergences associated with proper composite scoring rules and their applications
• Properization: constructing proper scoring rules via Bayes acts
• Metrics defined by Bregman divergences
• Bregman divergence and density integration
• Bregman divergences from comparative convexity
• Proper scoring rules for general decision models
• Penalized Bregman divergence estimation via coordinate descent
• Screening-based Bregman divergence estimation with NP-dimensionality
• Metrics defined by Bregman divergences. II

Cited in

(15)

• Asymptotic minimum scoring rule prediction
• Robust estimation of a location parameter with the integrated Hogg function
• Graph-based composite local Bregman divergences on discrete sample spaces
• Coherent conditional previsions and proper scoring rules
• Scoring Rules, Generalized Entropy, and Utility Maximization
• Affine invariant divergences associated with proper composite scoring rules and their applications
• Weighted Scoring Rules and Convex Risk Measures
• Strictly Proper Scoring Rules, Prediction, and Estimation
• Unbiased risk estimation and scoring rules
• Existence and uniqueness of proper scoring rules
• Proper scoring rules with arbitrary value functions
• A multidimensional objective prior distribution from a scoring rule
• A density power divergence measure to discriminate between generalized exponential and Weibull distributions
• Localizing Strictly Proper Scoring Rules
• The geometry of proper scoring rules

This page was built for publication: Proper scoring rules and Bregman divergence

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2405175)