Reversible Jump PDMP Samplers for Variable Selection

DOI10.1080/01621459.2022.2099402arXiv2010.11771OpenAlexW3093660365MaRDI QIDQ6185587FDOQ6185587

Authors: Augustin Chevallier, Paul Fearnhead, Matthew William Sutton 

Publication date: 8 January 2024

Published in: Journal of the American Statistical Association (Search for Journal in Brave)

Abstract: A new class of Markov chain Monte Carlo (MCMC) algorithms, based on simulating piecewise deterministic Markov processes (PDMPs), have recently shown great promise: they are non-reversible, can mix better than standard MCMC algorithms, and can use subsampling ideas to speed up computation in big data scenarios. However, current PDMP samplers can only sample from posterior densities that are differentiable almost everywhere, which precludes their use for model choice. Motivated by variable selection problems, we show how to develop reversible jump PDMP samplers that can jointly explore the discrete space of models and the continuous space of parameters. Our framework is general: it takes any existing PDMP sampler, and adds two types of trans-dimensional moves that allow for the addition or removal of a variable from the model. We show how the rates of these trans-dimensional moves can be calculated so that the sampler has the correct invariant distribution. Simulations show that the new samplers can mix better than standard MCMC algorithms. Our empirical results show they are also more efficient than gradient-based samplers that avoid model choice through use of continuous spike-and-slab priors which replace a point mass at zero for each parameter with a density concentrated around zero.

Full work available at URL: https://arxiv.org/abs/2010.11771

Recommendations

• Reversible jump MCMC
• Reversible jump MCMC to identify dropout mechanism in longitudinal data
• Reversible jump Markov chain Monte Carlo algorithms for Bayesian variable selection in logistic mixed models
• scientific article; zbMATH DE number 1762811
• A novel reversible jump algorithm for generalized linear models
• Reversible jump MCMC for nonparametric drift estimation for diffusion processes
• Reversible jump methods for generalised linear models and generalised linear mixed models
• A Bayesian lasso via reversible-jump MCMC
• A data-driven reversible jump for estimating a finite mixture of regression models
• Adaptive proposal construction for reversible jump MCMC

zbMATH Keywords

Monte CarloBayesian statisticsmodel choicebouncy particle samplerZigZag algorithm

Mathematics Subject Classification ID

Statistics (62-XX)

Cites Work

• MCMC using Hamiltonian dynamics
• The no-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• The horseshoe estimator for sparse signals
• Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables
• Nonparametric regression using Bayesian variable selection
• Analysis of a nonreversible Markov chain sampler.
• Bayesian Variable Selection in Linear Regression
• Sparsity information and regularization in the horseshoe and other shrinkage priors
• Random lasso
• Bayesian analysis of mixture models with an unknown number of components\,--\,an alternative to reversible jump methods.
• Spike and slab variable selection: frequentist and Bayesian strategies
• The zig-zag process and super-efficient sampling for Bayesian analysis of big data
• The Bouncy Particle Sampler: A Non-Reversible Rejection-Free Markov Chain Monte Carlo Method
• Coordinate sampler: a non-reversible Gibbs-like MCMC sampler
• Evolutionary stochastic search for Bayesian model exploration
• Variable selection for regression models
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• On the computational complexity of high-dimensional Bayesian variable selection
• Extended stochastic gradient Markov chain Monte Carlo for large-scale Bayesian variable selection
• A piecewise deterministic scaling limit of lifted Metropolis-Hastings in the Curie-Weiss model
• Piecewise deterministic Markov processes for continuous-time Monte Carlo
• Piecewise deterministic Markov processes for scalable Monte Carlo on restricted domains
• Scalable Importance Tempering and Bayesian Variable Selection
• Forward Event-Chain Monte Carlo: Fast Sampling by Randomness Control in Irreversible Markov Chains
• Sticky PDMP samplers for sparse and local inference problems
• A discrete bouncy particle sampler

Cited In (1)

• A novel reversible jump algorithm for generalized linear models

This page was built for publication: Reversible Jump PDMP Samplers for Variable Selection

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6185587)