Scalable Bayesian model averaging through local information propagation

DOI10.1080/01621459.2014.980908MaRDI QIDQ5367402zbMATH OpenOpenAlexFDO

Authors Li Ma

Publication date 13 October 2017

Published in Journal of the American Statistical Association (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1403.2397

zbMATH Keywords

(generalized) linear modelsBayesian model choicehidden Markov modelhigh-dimensional inferenceMarkov processparticle filtersequential importance samplingvariable selection

Mathematics Subject Classification ID

Bayesian inference (62F15) Linear regression; mixed models (62J05) Generalized linear models (logistic models) (62J12) Markov processes: estimation; hidden Markov models (62M05) Monte Carlo methods (65C05)

Abstract: We show that a probabilistic version of the classical forward-stepwise variable inclusion procedure can serve as a general data-augmentation scheme for model space distributions in (generalized) linear models. This latent variable representation takes the form of a Markov process, thereby allowing information propagation algorithms to be applied for sampling from model space posteriors. In particular, we propose a sequential Monte Carlo method for achieving effective unbiased Bayesian model averaging in high-dimensional problems, utilizing proposal distributions constructed using local information propagation. We illustrate our method---called LIPS for local information propagation based sampling---through real and simulated examples with dimensionality ranging from 15 to 1,000, and compare its performance in estimating posterior inclusion probabilities and in out-of-sample prediction to those of several other methods---namely, MCMC, BAS, iBMA, and LASSO. In addition, we show that the latent variable representation can also serve as a modeling tool for specifying model space priors that account for knowledge regarding model complexity and conditional inclusion relationships.

Recommendations

• scientific article; zbMATH DE number 1522700
• Scalable Importance Tempering and Bayesian Variable Selection
• Adaptive sampling for Bayesian variable selection
• Bayesian Model Averaging for Linear Regression Models
• Scalable variational inference for Bayesian variable selection in regression, and its accuracy in genetic association studies

Cited in

(4)

• Adaptive random neighbourhood informed Markov chain Monte Carlo for high-dimensional Bayesian variable selection
• Particle EM for variable selection
• Model-averaged _1 regularization using Markov chain Monte Carlo model composition
• Bayesian multiplicity correction in the probabilistic forward stepwise framework

This page was built for publication: Scalable Bayesian model averaging through local information propagation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5367402)