The correlated pseudomarginal method

DOI10.48550/ARXIV.1511.04992MaRDI QIDQ122164zbMATH OpenOpenAlexWikidataFDO

Authors George Deligiannidis, George Deligiannidis, Arnaud Doucet, Michael K. Pitt

Publication date 16 November 2015

Published in Journal of the Royal Statistical Society Series B: Statistical Methodology (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1511.04992

zbMATH Keywords

asymptotic posterior normalitycorrelated random numbersintractable likelihoodMetropolis-Hastings algorithmparticle filterrandom-effects modelweak convergence

Mathematics Subject Classification ID

Central limit and other weak theorems (60F05) Bayesian inference (62F15) Monte Carlo methods (65C05)

Abstract: The pseudo-marginal algorithm is a popular variant of the Metropolis--Hastings scheme which allows us to sample asymptotically from a target probability density $pi$, when we are only able to estimate an unnormalized version of $pi$ pointwise unbiasedly. It has found numerous applications in Bayesian statistics as there are many scenarios where the likelihood function is intractable but can be estimated unbiasedly using Monte Carlo samples. Using many samples will typically result in averages computed under this chain with lower asymptotic variances than the corresponding averages that use fewer samples. For a fixed computing time, it has been shown in several recent contributions that an efficient implementation of the pseudo-marginal method requires the variance of the log-likelihood ratio estimator appearing in the acceptance probability of the algorithm to be of order 1, which in turn usually requires scaling the number $N$ of Monte Carlo samples linearly with the number $T$ of data points. We propose a modification of the pseudo-marginal algorithm, termed the correlated pseudo-marginal algorithm, which is based on a novel log-likelihood ratio estimator computed using the difference of two positively correlated log-likelihood estimators. We show that the parameters of this scheme can be selected such that the variance of this estimator is order $1$ as $N,Tightarrowinfty$ whenever $N/Tightarrow0$. By combining these results with the Bernstein-von Mises theorem, we provide an analysis of the performance of the correlated pseudo-marginal algorithm in the large $T$ regime. In our numerical examples, the efficiency of computations is increased relative to the standard pseudo-marginal algorithm by more than 20 fold for values of $T$ of a few hundreds to more than 100 fold for values of $T$ of around 10,000-20,000.

Recommendations

• The pseudo-marginal approach for efficient Monte Carlo computations
• Large-sample asymptotics of the pseudo-marginal method
• Subsampling MCMC -- an introduction for the survey statistician
• On the efficiency of pseudo-marginal random walk Metropolis algorithms
• Convergence properties of pseudo-marginal Markov chain Monte Carlo algorithms

Cited in

(48)

• The pseudo-marginal approach for efficient Monte Carlo computations
• cPseudoMaRg
• Biased online parameter inference for state-space models
• Bayesian estimation of dynamic asset pricing models with informative observations
• Data-cloning SMC\(^2\): a global optimizer for maximum likelihood estimation of latent variable models
• An adaptive MCMC method for Bayesian variable selection in logistic and accelerated failure time regression models
• Particle methods for stochastic differential equation mixed effects models
• Bayesian parameter inference for partially observed stochastic differential equations driven by fractional Brownian motion
• Augmented pseudo-marginal Metropolis-Hastings for partially observed diffusion processes
• Gaussian process enhanced semi-automatic approximate Bayesian computation: parameter inference in a stochastic differential equation system for chemotaxis
• Optimal scaling of random walk Metropolis algorithms using Bayesian large-sample asymptotics
• Statistic selection and MCMC for differentially private Bayesian estimation
• Unbiased Markov chain Monte Carlo for intractable target distributions
• A flexible particle Markov chain Monte Carlo method
• Subsampling sequential Monte Carlo for static Bayesian models
• Efficient Bayesian model choice for partially observed processes: with application to an experimental transmission study of an infectious disease
• Subsampling MCMC -- an introduction for the survey statistician
• Efficient \(\mathrm{SMC}^2\) schemes for stochastic kinetic models
• Computationally efficient Bayesian estimation of high-dimensional Archimedean copulas with discrete and mixed margins
• Correlated pseudo-marginal schemes for time-discretised stochastic kinetic models
• scientific article; zbMATH DE number 4151591 (Why is no real title available?)
• Bayesian static parameter estimation for partially observed diffusions via multilevel Monte Carlo
• The Block-Poisson Estimator for Optimally Tuned Exact Subsampling MCMC
• Smoothing with couplings of conditional particle filters
• Pseudo-marginal Hamiltonian Monte Carlo
• Hamiltonian Monte Carlo with energy conserving subsampling
• Advanced Multilevel Monte Carlo Methods
• Accelerating Bayesian inference for stochastic epidemic models using incidence data
• Accelerating inference for stochastic kinetic models
• A point mass proposal method for Bayesian state-space model fitting
• Ensemble MCMC: accelerating pseudo-marginal MCMC for state space models using the ensemble Kalman filter
• Sequentially guided MCMC proposals for synthetic likelihoods and correlated synthetic likelihoods
• Metropolis–Hastings via Classification
• A Statistical Recurrent Stochastic Volatility Model for Stock Markets
• Bayesian Inference Using Synthetic Likelihood: Asymptotics and Adjustments
• Mixture copulas with discrete margins and their application to imbalanced data
• Computing Bayes: from then `til now
• Approximating Bayes in the 21st century
• Bayesian Inference in Common Microeconometric Models With Massive Datasets by Double Marginalized Subsampling
• Generalized Bayesian likelihood-free inference
• Estimating the Hawkes Process From a Discretely Observed Sample Path
• Auxiliary MCMC samplers for parallelisable inference in high-dimensional latent dynamical systems
• The Block-Correlated Pseudo Marginal Sampler for State Space Models
• Solving geophysical inversion problems with intractable likelihoods: linearized Gaussian approximations versus the correlated pseudo-marginal method
• Analysing symbolic data by pseudo-marginal methods
• Simulation of infinite-dimensional diffusion bridges
• Particle MCMC and the correlated particle hybrid sampler for state space models
• Efficient inference for stochastic differential equation mixed-effects models using correlated particle pseudo-marginal algorithms

This page was built for publication: The correlated pseudomarginal method

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q122164)