Subsampling MCMC -- an introduction for the survey statistician
From MaRDI portal
Publication:2316968
Abstract: The rapid development of computing power and efficient Markov Chain Monte Carlo (MCMC) simulation algorithms have revolutionized Bayesian statistics, making it a highly practical inference method in applied work. However, MCMC algorithms tend to be computationally demanding, and are particularly slow for large datasets. Data subsampling has recently been suggested as a way to make MCMC methods scalable on massively large data, utilizing efficient sampling schemes and estimators from the survey sampling literature. These developments tend to be unknown by many survey statisticians who traditionally work with non-Bayesian methods, and rarely use MCMC. Our article explains the idea of data subsampling in MCMC by reviewing one strand of work, Subsampling MCMC, a so called pseudo-marginal MCMC approach to speeding up MCMC through data subsampling. The review is written for a survey statistician without previous knowledge of MCMC methods since our aim is to motivate survey sampling experts to contribute to the growing Subsampling MCMC literature.
The paper presents the pseudo-marginal approach to subsampling in Markov chain Monte Carlo from the perspective of a survey statistician. Several effective control variates for variance reduction of the likelihood estimator which make subsampling MCMC scalable to large datasets have been reviewed. Methods for correlating the subsamples over the MCMC iterations, ultimately leading to algorithms that allow much more variable likelihood estimators, have also been presented. Much of the focus was given to unbiased estimators of the log-likelihood and methods for bias-correction of the resulting likelihood estimators.
Recommendations
- Subsampling sequential Monte Carlo for static Bayesian models
- Informed sub-sampling MCMC: approximate Bayesian inference for large datasets
- Speeding Up MCMC by Efficient Data Subsampling
- Subsample distribution distance and McMC convergence
- Applicability of subsampling bootstrap methods in Markov chain Monte Carlo
- Subsampling the Gibbs sampler: variance reduction
- Speeding up MCMC by Delayed Acceptance and Data Subsampling
- A subsampling approach for Bayesian model selection
Cites work
- An introduction to variational methods for graphical models
- Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion)
- Asymptotic Statistics
- Bayesian interference based only on simulated likelihood particle filter analysis of dynamic economic models
- Calibration Estimators in Survey Sampling
- Dependence modeling with copulas
- Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator
- Equation of state calculations by fast computing machines
- Expectation propagation as a way of life: a framework for Bayesian inference on partitioned data
- Hamiltonian Monte Carlo with energy conserving subsampling
- Handbook of Markov Chain Monte Carlo
- scientific article; zbMATH DE number 1666084 (Why is no real title available?)
- scientific article; zbMATH DE number 3982257 (Why is no real title available?)
- scientific article; zbMATH DE number 2106098 (Why is no real title available?)
- scientific article; zbMATH DE number 6781368 (Why is no real title available?)
- scientific article; zbMATH DE number 3196612 (Why is no real title available?)
- Informed sub-sampling MCMC: approximate Bayesian inference for large datasets
- MCMC using Hamiltonian dynamics
- Merging MCMC subposteriors through Gaussian-process approximations
- Model assisted survey sampling.
- Monte Carlo sampling methods using Markov chains and their applications
- Noisy Monte Carlo: convergence of Markov chains with approximate transition kernels
- On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods
- On some properties of Markov chain Monte Carlo simulation methods based on the particle filter
- On the efficiency of pseudo-marginal random walk Metropolis algorithms
- Speeding up MCMC by Delayed Acceptance and Data Subsampling
- Speeding Up MCMC by Efficient Data Subsampling
- Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images
- Subsampling sequential Monte Carlo for static Bayesian models
- The Block-Poisson Estimator for Optimally Tuned Exact Subsampling MCMC
- The Bouncy Particle Sampler: A Non-Reversible Rejection-Free Markov Chain Monte Carlo Method
- The correlated pseudomarginal method
- The no-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo
- The pseudo-marginal approach for efficient Monte Carlo computations
- The zig-zag process and super-efficient sampling for Bayesian analysis of big data
- Unbiased estimation with square root convergence for SDE models
- Unbiased multi-step estimators for the Monte Carlo evaluation of certain functional integrals
- Weak convergence and optimal scaling of random walk Metropolis algorithms
Cited in
(17)- The correlated pseudomarginal method
- Subsampling the Gibbs sampler: variance reduction
- A subsampling approach for Bayesian model selection
- Properties of the bridge sampler with a focus on splitting the MCMC sample
- Subsampling sequential Monte Carlo for static Bayesian models
- Informed sub-sampling MCMC: approximate Bayesian inference for large datasets
- Selection sampling from large data sets for targeted inference in mixture modeling
- Distributed Bayesian Inference in Linear Mixed-Effects Models
- The Block-Poisson Estimator for Optimally Tuned Exact Subsampling MCMC
- Hamiltonian Monte Carlo with energy conserving subsampling
- Speeding Up MCMC by Efficient Data Subsampling
- Applicability of subsampling bootstrap methods in Markov chain Monte Carlo
- Stochastic gradient Markov chain Monte Carlo
- On randomized sketching algorithms and the Tracy-Widom law
- An efficient adaptive MCMC algorithm for pseudo-Bayesian quantum tomography
- Emerging directions in Bayesian computation
- Perturbation analysis of Markov chain Monte Carlo for graphical models
This page was built for publication: Subsampling MCMC -- an introduction for the survey statistician
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2316968)
