Probability aggregation in time-series: dynamic hierarchical modeling of sparse expert beliefs

DOI10.1214/14-AOAS739MaRDI QIDQ400690zbMATH OpenOpenAlexFDO

Authors Ville A. Satopää, Shane T. Jensen, Barbara A. Mellers, Philip E. Tetlock, Lyle H. Ungar

Publication date 22 August 2014

Published in The Annals of Applied Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1408.0087

zbMATH Keywords

time seriesbias estimationcalibrationdynamic linear modelexpert forecasthierarchical modelingprobability aggregationsubjective probability

Mathematics Subject Classification ID

Time series, auto-correlation, regression, etc. in statistics (GARCH) (62M10) Applications of statistics to social sciences (62P25)

Abstract: Most subjective probability aggregation procedures use a single probability judgment from each expert, even though it is common for experts studying real problems to update their probability estimates over time. This paper advances into unexplored areas of probability aggregation by considering a dynamic context in which experts can update their beliefs at random intervals. The updates occur very infrequently, resulting in a sparse data set that cannot be modeled by standard time-series procedures. In response to the lack of appropriate methodology, this paper presents a hierarchical model that takes into account the expert's level of self-reported expertise and produces aggregate probabilities that are sharp and well calibrated both in- and out-of-sample. The model is demonstrated on a real-world data set that includes over 2300 experts making multiple probability forecasts over two years on different subsets of 166 international political events.

Recommendations

• Combining probability distributions: A critique and an annotated bibliography
• Aggregating disparate estimates of chance
• Combining probability distributions: a critique and an annotated bibliography. With comments, and a rejoinder by the authors
• Publication:3484159
• Aggregating large sets of probabilistic forecasts by weighted coherent adjustment

Cites work

• scientific article; zbMATH DE number 48317 (Why is no real title available?)
• scientific article; zbMATH DE number 1865745 (Why is no real title available?)
• scientific article; zbMATH DE number 2107164 (Why is no real title available?)
• scientific article; zbMATH DE number 3026527 (Why is no real title available?)
• A weakly informative default prior distribution for logistic and other regression models
• Bayesian data analysis.
• Combining predictive distributions
• Combining probability distributions: A critique and an annotated bibliography
• Comments on: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds
• Low Moments for Small Samples: A Comparative Study of Order Statistics
• On Gibbs sampling for state space models
• Probability aggregation in time-series: dynamic hierarchical modeling of sparse expert beliefs
• Probability aggregation methods in geoscience
• Rejoinder on: Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds
• Slice sampling. (With discussions and rejoinder)
• Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
• Two reasons to make aggregated probability forecasts more extreme

Cited in

(3)

• Probability aggregation in time-series: dynamic hierarchical modeling of sparse expert beliefs
• Aggregating predictions from experts: a review of statistical methods, experiments, and applications
• Multivariate Bayesian predictive synthesis in macroeconomic forecasting

Describes a project that uses

Uses Software

• BayesDA

This page was built for publication: Probability aggregation in time-series: dynamic hierarchical modeling of sparse expert beliefs

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q400690)