Irrational Exuberance: Correcting Bias in Probability Estimates

DOI10.1080/01621459.2020.1787175MaRDI QIDQ5881099zbMATH OpenOpenAlexFDO

Authors Gareth M. James, Peter Radchenko

Publication date 9 March 2023

Published in Journal of the American Statistical Association (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1910.13570

zbMATH Keywords

empirical Bayesexcess certaintyselection biasTweedie's formula

Mathematics Subject Classification ID

Bayesian inference (62F15) Applications of statistics to actuarial sciences and financial mathematics (62P05)

Abstract: We consider the common setting where one observes probability estimates for a large number of events, such as default risks for numerous bonds. Unfortunately, even with unbiased estimates, selecting events corresponding to the most extreme probabilities can result in systematically underestimating the true level of uncertainty. We develop an empirical Bayes approach "Excess Certainty Adjusted Probabilities" (ECAP), using a variant of Tweedie's formula, which updates probability estimates to correct for selection bias. ECAP is a flexible non-parametric method, which directly estimates the score function associated with the probability estimates, so it does not need to make any restrictive assumptions about the prior on the true probabilities. ECAP also works well in settings where the probability estimates are biased. We demonstrate through theoretical results, simulations, and an analysis of two real world data sets, that ECAP can provide significant improvements over the original probability estimates.

Recommendations

• Default probability estimation in small samples -- with an application to sovereign bonds
• Tweedie’s Formula and Selection Bias
• Measuring and adjusting for overconfidence
• Two reasons to make aggregated probability forecasts more extreme
• Quantifying and Correcting the Bias in Estimated Risk Measures

Cites work

• Adapting to unknown sparsity by controlling the false discovery rate
• Comparison of linear shrinkage estimators of a large covariance matrix in normal and non-normal distributions
• Covariance regularization by thresholding
• Data Analysis Using Stein's Estimator and its Generalizations
• Empirical Bayes methods in classical and Bayesian inference
• Estimation with quadratic loss.
• False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters
• General maximum likelihood empirical Bayes estimation of normal means
• High-dimensional additive modeling
• scientific article; zbMATH DE number 3124366 (Why is no real title available?)
• scientific article; zbMATH DE number 5654889 (Why is no real title available?)
• scientific article; zbMATH DE number 3229907 (Why is no real title available?)
• Ideal spatial adaptation by wavelet shrinkage
• Making the Cut: Improved Ranking and Selection for Large-Scale Inference
• Nonlinear shrinkage estimation of large-dimensional covariance matrices
• Nonparametric empirical Bayes and compound decision approaches to estimation of a high-dimensional vector of normal means
• Optimal rates of convergence for nonparametric estimators
• Philosophy and the practice of Bayesian statistics
• Tweedie’s Formula and Selection Bias

This page was built for publication: Irrational Exuberance: Correcting Bias in Probability Estimates

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5881099)