Merging MCMC subposteriors through Gaussian-process approximations
From MaRDI portal
Publication:1631561
Abstract: Markov chain Monte Carlo (MCMC) algorithms have become powerful tools for Bayesian inference. However, they do not scale well to large-data problems. Divide-and-conquer strategies, which split the data into batches and, for each batch, run independent MCMC algorithms targeting the corresponding subposterior, can spread the computational burden across a number of separate workers. The challenge with such strategies is in recombining the subposteriors to approximate the full posterior. By creating a Gaussian-process approximation for each log-subposterior density we create a tractable approximation for the full posterior. This approximation is exploited through three methodologies: firstly a Hamiltonian Monte Carlo algorithm targeting the expectation of the posterior density provides a sample from an approximation to the posterior; secondly, evaluating the true posterior at the sampled points leads to an importance sampler that, asymptotically, targets the true posterior expectations; finally, an alternative importance sampler uses the full Gaussian-process distribution of the approximation to the log-posterior density to re-weight any initial sample and provide both an estimate of the posterior expectation and a measure of the uncertainty in it.
Recommendations
- Merging the components of a finite mixture using posterior probabilities
- Accelerating pseudo-marginal MCMC using Gaussian processes
- Posterior consistency for Gaussian process approximations of Bayesian posterior distributions
- Subsampling sequential Monte Carlo for static Bayesian models
- Mixing of MCMC algorithms
- Sequentially allocated merge-split samplers for conjugate Bayesian nonparametric models
- Mixture models with an unknown number of components via a new posterior split-merge MCMC algorithm
- scientific article; zbMATH DE number 1843059
- Particle Gibbs split-merge sampling for Bayesian inference in mixture models
Cites work
- A unifying view of sparse approximate Gaussian process regression
- Bayesian Inference in Econometric Models Using Monte Carlo Integration
- Consistency and fluctuations for stochastic gradient Langevin dynamics
- Gaussian processes for machine learning.
- scientific article; zbMATH DE number 3583145 (Why is no real title available?)
- scientific article; zbMATH DE number 1350773 (Why is no real title available?)
- scientific article; zbMATH DE number 849923 (Why is no real title available?)
- MCMC using Hamiltonian dynamics
- Nonparametric density estimation with a parametric start
- Optimal Scaling of Discrete Approximations to Langevin Diffusions
- Optimal tuning of the hybrid Monte Carlo algorithm
- Riemann manifold Langevin and Hamiltonian Monte Carlo methods. With discussion and authors' reply
- Sparse on-line Gaussian processes
- The no-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo
- Weak convergence and optimal scaling of random walk Metropolis algorithms
Cited in
(18)- Comparing consensus Monte Carlo strategies for distributed Bayesian computation
- Challenges in Markov chain Monte Carlo for Bayesian neural networks
- Subsampling MCMC -- an introduction for the survey statistician
- Distributed Bayesian matrix factorization with limited communication
- Speeding up MCMC by Delayed Acceptance and Data Subsampling
- Scalable Bayes via barycenter in Wasserstein space
- Hamiltonian Monte Carlo with energy conserving subsampling
- Speeding Up MCMC by Efficient Data Subsampling
- Stochastic gradient Markov chain Monte Carlo
- Reflections on Bayesian inference and Markov chain Monte Carlo
- Divide-and-conquer Metropolis-Hastings samplers with matched samples
- Emerging directions in Bayesian computation
- SwISS: a scalable Markov chain Monte Carlo divide-and-conquer strategy
- Bayesian Inference in Common Microeconometric Models With Massive Datasets by Double Marginalized Subsampling
- Fast post-process Bayesian inference with variational sparse Bayesian quadrature
- Energy distance-based subsampling Markov chain Monte Carlo
- Analysing symbolic data by pseudo-marginal methods
- Parallel MCMC via machine learning classification and a Kullback-Leibler divergence criterion
This page was built for publication: Merging MCMC subposteriors through Gaussian-process approximations
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1631561)
