On the marginal likelihood and cross-validation

DOI10.1093/BIOMET/ASZ077MaRDI QIDQ5113024zbMATHOpenAlexWikidataFDO

Authors Ed Fong, Chris Holmes

Publication date 9 June 2020

Published in Biometrika (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1905.08737

zbMATH Keywords

cross-validationmarginal likelihoodprequential scoring

Mathematics Subject Classification ID

Foundations and philosophical topics in statistics (62A01) Sufficient statistics and fields (62B05)

Abstract: In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-fold partitioning or leave-$p$-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-$p$-out cross-validation averaged over all values of $p$ and all held-out test sets when using the log posterior predictive probability as the scoring rule. Moreover, the log posterior predictive is the only coherent scoring rule under data exchangeability. This offers new insight into the marginal likelihood and cross-validation and highlights the potential sensitivity of the marginal likelihood to the choice of the prior. We suggest an alternative approach using cumulative cross-validation following a preparatory training phase. Our work has connections to prequential analysis and intrinsic Bayes factors but is motivated through a different course.

Recommendations

• Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities
• Information criteria and cross validation for Bayesian inference in regular and singular cases
• Cross-validation prior choice in Bayesian probit regression with many covariates
• A comparison of marginal likelihood computation methods
• Bayesian model selection and model averaging

Cited in

(15)

• Marginal Likelihood Computation for Model Selection and Hypothesis Testing: An Extensive Review
• Bayesian artificial neural networks for frontier efficiency analysis
• Adaptation of the tuning parameter in general Bayesian inference with robust divergence
• Evidential Calibration of Confidence Intervals
• Marginal Likelihood Estimation with the Cross-Entropy Method
• A general approximation to nested Bayes factors with informed priors
• Assessment of generalised Bayesian structural equation models for continuous and binary data
• Combining data envelopment analysis and stochastic frontiers via a LASSO prior
• Maximum likelihood estimation and uncertainty quantification for Gaussian process approximation of deterministic functions
• Fast Cross-validation for Multi-penalty High-dimensional Ridge Regression
• Asymptotic Bounds for Smoothness Parameter Estimates in Gaussian Process Interpolation
• Asymptotic Properties of Adaptive Likelihood Weights by Cross-Validation
• Bayesian learning in performance. Is there any?
• Priors in Bayesian Deep Learning: A Review
• The no-free-lunch theorems of supervised learning

This page was built for publication: On the marginal likelihood and cross-validation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5113024)