Reproducible model selection using bagged posteriors

DOI10.1214/21-BA1301arXiv2007.14845OpenAlexW4210935632MaRDI QIDQ6122016FDOQ6122016

Authors: Jonathan H. Huggins, Jeffrey W. Miller 

Publication date: 27 February 2024

Published in: Bayesian Analysis (Search for Journal in Brave)

Abstract: Bayesian model selection is premised on the assumption that the data are generated from one of the postulated models. However, in many applications, all of these models are incorrect (that is, there is misspecification). When the models are misspecified, two or more models can provide a nearly equally good fit to the data, in which case Bayesian model selection can be highly unstable, potentially leading to self-contradictory findings. To remedy this instability, we propose to use bagging on the posterior distribution ("BayesBag") -- that is, to average the posterior model probabilities over many bootstrapped datasets. We provide theoretical results characterizing the asymptotic behavior of the posterior and the bagged posterior in the (misspecified) model selection setting. We empirically assess the BayesBag approach on synthetic and real-world data in (i) feature selection for linear regression and (ii) phylogenetic tree reconstruction. Our theory and experiments show that, when all models are misspecified, BayesBag (a) provides greater reproducibility and (b) places posterior mass on optimal models more reliably, compared to the usual Bayesian posterior; on the other hand, under correct specification, BayesBag is slightly more conservative than the usual posterior, in the sense that BayesBag posterior probabilities tend to be slightly farther from the extremes of zero and one. Overall, our results demonstrate that BayesBag provides an easy-to-use and widely applicable approach that improves upon Bayesian model selection by making it more stable and reproducible.

Full work available at URL: https://arxiv.org/abs/2007.14845

zbMATH Keywords

bootstrapasymptoticsBayesian model averagingmodel misspecificationstabilitybagging

Mathematics Subject Classification ID

Statistics (62-XX)

Cites Work

• Bagging predictors
• Analyzing bagging
• Limiting Behavior of Posterior Distributions when the Model is Incorrect
• Discussion of big Bayes stories and BayesBag
• Models as approximations. II. A model-free theory of parametric regression
• Information-theoretic asymptotics of Bayes methods
• Bayesian selection of misspecified models is overconfident and may cause spurious posterior probabilities for phylogenetic trees
• Models as approximations. I. Consequences illustrated with linear regression
• Comparing and Weighting Imperfect Models Using D-Probabilities

Cited In (1)

• Optimal experimental design: formulations and computations

This page was built for publication: Reproducible model selection using bagged posteriors

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6122016)