Efficient MCMC for Gibbs random fields using pre-computation

DOI10.1214/18-EJS1504MaRDI QIDQ1711571zbMATH OpenOpenAlexFDO

Authors Aidan Boland, Nial Friel, Florian Maire

Publication date 18 January 2019

Published in Electronic Journal of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1710.04093, https://projecteuclid.org/euclid.ejs/1544670254

zbMATH Keywords

exponential random graph modelsGibbs random fieldsMCMC

Mathematics Subject Classification ID

Bayesian inference (62F15) Random fields; image analysis (62M40)

Abstract: Bayesian inference of Gibbs random fields (GRFs) is often referred to as a doubly intractable problem, since the likelihood function is intractable. The exploration of the posterior distribution of such models is typically carried out with a sophisticated Markov chain Monte Carlo (MCMC) method, the exchange algorithm (Murray et al., 2006), which requires simulations from the likelihood function at each iteration. The purpose of this paper is to consider an approach to dramatically reduce this computational overhead. To this end we introduce a novel class of algorithms which use realizations of the GRF model, simulated offline, at locations specified by a grid that spans the parameter space. This strategy speeds up dramatically the posterior inference, as illustrated on several examples. However, using the pre-computed graphs introduces a noise in the MCMC algorithm, which is no longer exact. We study the theoretical behaviour of the resulting approximate MCMC algorithm and derive convergence bounds using a recent theoretical development on approximate MCMC methods.

Review text

The paper under review deals with the problem of reducing the computational overhead in exploration of the doubly-intractable posterior distribution of the Gibbs random field (GRF) models, which is carried out with the exchange algorithm [\textit{I. Murray}, \textit{Z. Ghahramani} and \textit{D. J. C. MacKay}, ``MCMC for doubly-intractable distributions, in: Proceedings of the 22nd annual conference on uncertainty in artificial intelligence, UAI'06. Arlington, VA: AUAI Press. 359--366 (2006)]. That algorithm is a Markov chain Monte Carlo (MCMC) method, which extends the Metropolis-Hastings algorithm to situations where the likelihood is intractable. The authors introduce a new class of algorithms that use realizations of the GRF model, simulated offline. It is illustrated on examples that this strategy speeds up the posterior inference. Moreover, theoretical behavior of the resulting approximate MCMC algorithm is studied and convergence bounds are derived. In conclusion, the authors note that their ``method would be impractical for very high dimensions and discuss related open problems.

Recommendations

• Model comparison for Gibbs random fields using noisy reversible jump Markov chain Monte Carlo
• On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods
• Markov-chain monte carlo: Some practical implications of theoretical results
• ABC likelihood-free methods for model choice in Gibbs random fields
• Generalised Gibbs sampler and multigrid Monte Carlo for Bayesian computation

Cites work

• A Bayesian Reassessment of Nearest-Neighbor Classification
• A note on Metropolis-Hastings kernels for general state spaces
• A Stochastic Approximation Method
• Approximate Bayesian computational methods
• Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion)
• Equation of state calculations by fast computing machines
• Exact sampling with coupled Markov chains and applications to statistical mechanics
• Exploiting multi-core architectures for reduced-variance estimation with intractable likelihoods
• Gibbs measures and phase transitions.
• scientific article; zbMATH DE number 3513115 (Why is no real title available?)
• scientific article; zbMATH DE number 840151 (Why is no real title available?)
• scientific article; zbMATH DE number 6781368 (Why is no real title available?)
• Instability, sensitivity, and degeneracy of discrete exponential families
• Mixing time of exponential random graphs
• Noisy Hamiltonian Monte Carlo for Doubly Intractable Distributions
• Noisy Monte Carlo: convergence of Markov chains with approximate transition kernels
• Optimum Monte-Carlo sampling using Markov chains
• Pre-processing for approximate Bayesian computation in image analysis
• Scalable Bayesian inference for the inverse temperature of a hidden Potts model
• Sequential Monte Carlo Samplers
• Simulating normalizing constants: From importance sampling to bridge sampling to path sampling
• Speeding up MCMC by Delayed Acceptance and Data Subsampling
• Stability of noisy Metropolis-Hastings
• The pseudo-marginal approach for efficient Monte Carlo computations

Cited in

(9)

• Approximate Bayesian inference for hierarchical Gaussian Markov random field models
• On Russian roulette estimates for Bayesian inference with doubly-intractable likelihoods
• Model comparison for Gibbs random fields using noisy reversible jump Markov chain Monte Carlo
• Scalable Bayesian inference for the inverse temperature of a hidden Potts model
• Bayesian indirect inference for models with intractable normalizing functions
• Preconditioning Markov Chain Monte Carlo Simulations Using Coarse-Scale Models
• Finding our way in the dark: approximate MCMC for approximate Bayesian methods
• Warped gradient-enhanced Gaussian process surrogate models for exponential family likelihoods with intractable normalizing constants
• Approximations in the Homogeneous Ising Model with Application to Scene Analysis

This page was built for publication: Efficient MCMC for Gibbs random fields using pre-computation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1711571)