Approximating cross-validatory predictive evaluation in Bayesian latent variable models with integrated IS and WAIC
From MaRDI portal
Publication:2628889
Abstract: A natural method for approximating out-of-sample predictive evaluation is leave-one-out cross-validation (LOOCV) --- we alternately hold out each case from a full data set and then train a Bayesian model using Markov chain Monte Carlo (MCMC) without the held-out; at last we evaluate the posterior predictive distribution of all cases with their actual observations. However, actual LOOCV is time-consuming. This paper introduces two methods, namely iIS and iWAIC, for approximating LOOCV with only Markov chain samples simulated from a posterior based on a extit{full} data set. iIS and iWAIC aim at improving the approximations given by importance sampling (IS) and WAIC in Bayesian models with possibly correlated latent variables. In iIS and iWAIC, we first integrate the predictive density over the distribution of the latent variables associated with the held-out without reference to its observation, then apply IS and WAIC approximations to the integrated predictive density. We compare iIS and iWAIC with other approximation methods in three real data examples that respectively use mixture models, models with correlated spatial effects, and a random effect logistic model. Our empirical results show that iIS and iWAIC give substantially better approximates than non-integrated IS and WAIC and other methods.
Recommendations
- scientific article; zbMATH DE number 6617268
- Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
- Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models
- Conditional vs marginal estimation of the predictive loss of hierarchical models using WAIC and cross-validation
- Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities
Cites work
- scientific article; zbMATH DE number 720676 (Why is no real title available?)
- scientific article; zbMATH DE number 947414 (Why is no real title available?)
- A STATISTICAL PARADOX
- A new approach to Bayesian hypothesis testing
- A survey of Bayesian predictive methods for model assessment, selection and comparison
- Algebraic Geometry and Statistical Learning Theory
- Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory
- Bayes Factors
- Bayesian Hypothesis Testing: A Reference Approach
- Bayesian Inference in Econometric Models Using Monte Carlo Integration
- Bayesian Measures of Model Complexity and Fit
- Bayesian Model Assessment and Comparison Using Cross-Validation Predictive Densities
- Bayesian analysis of stochastic volatility models
- Bayesian hypothesis testing in latent variable models
- Bayesian learning for neural networks
- Bayesian predictive information criterion for the evaluation of hierarchical Bayesian and empirical Bayes models
- Case-deletion importance sampling estimators: central limit theorems and related results
- Density Estimation With Confidence Sets Exemplified by Superclusters and Voids in the Galaxies
- Deviance information criteria for missing data models
- Deviance information criterion for latent variable models and misspecified models
- Difficulty of selecting among multilevel models using predictive accuracy
- Equations of states in singular statistical estimation
- GPstuff: Bayesian modeling with Gaussian processes
- Hierarchical Spatio-Temporal Mapping of Disease Rates
- Identifying outliers in Bayesian hierarchical models: a simulation-based approach
- Importance re-sampling MCMC for cross-validation in inverse problems
- Marginal Likelihood from the Gibbs Output
- Monte Carlo strategies in scientific computing
- On the Variability of Case-Deletion Importance Sampling Weights in the Bayesian Linear Model
- Penalized loss functions for Bayesian model comparison
- Properties of intrinsic and fractional Bayes factors
- Simulating normalizing constants: From importance sampling to bridge sampling to path sampling
- Stochastic volatility modelling in continuous time with general marginal distributions: inference, prediction and model selection
- The Intrinsic Bayes Factor for Model Selection and Prediction
- Understanding predictive information criteria for Bayesian models
Cited in
(8)- Prediction scoring of data-driven discoveries for reproducible research
- Efficient leave-one-out cross-validation for Bayesian non-factorized normal and Student-t models
- Bayesian model selection in linear mixed models for longitudinal data
- Model selection for Bayesian linear mixed models with longitudinal data: Sensitivity to the choice of priors
- Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC
- Bayesian model selection for multilevel mediation models
- Conditional vs marginal estimation of the predictive loss of hierarchical models using WAIC and cross-validation
- Bayesian comparison of latent variable models: conditional versus marginal likelihoods
This page was built for publication: Approximating cross-validatory predictive evaluation in Bayesian latent variable models with integrated IS and WAIC
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2628889)
