Ensemble transport adaptive importance sampling

DOI10.1137/17M1114867zbMATH Open1422.62104arXiv1508.01132WikidataQ127984008 ScholiaQ127984008MaRDI QIDQ5228364FDOQ5228364

Authors: S. L. Cotter, Paul T. Russell, C. J. Cotter 

Publication date: 12 August 2019

Published in: SIAM/ASA Journal on Uncertainty Quantification (Search for Journal in Brave)

Abstract: Markov chain Monte Carlo methods are a powerful and commonly used family of numerical methods for sampling from complex probability distributions. As applications of these methods increase in size and complexity, the need for efficient methods increases. In this paper, we present a particle ensemble algorithm. At each iteration, an importance sampling proposal distribution is formed using an ensemble of particles. A stratified sample is taken from this distribution and weighted under the posterior, a state-of-the-art ensemble transport resampling method is then used to create an evenly weighted sample ready for the next iteration. We demonstrate that this ensemble transport adaptive importance sampling (ETAIS) method outperforms MCMC methods with equivalent proposal distributions for low dimensional problems, and in fact shows better than linear improvements in convergence rates with respect to the number of ensemble members. We also introduce a new resampling strategy, multinomial transformation (MT), which while not as accurate as the ensemble transport resampler, is substantially less costly for large ensemble sizes, and can then be used in conjunction with ETAIS for complex problems. We also focus on how algorithmic parameters regarding the mixture proposal can be quickly tuned to optimise performance. In particular, we demonstrate this methodology's superior sampling for multimodal problems, such as those arising from inference for mixture models, and for problems with expensive likelihoods requiring the solution of a differential equation, for which speed-ups of orders of magnitude are demonstrated. Likelihood evaluations of the ensemble could be computed in a distributed manner, suggesting that this methodology is a good candidate for parallel Bayesian computations.

Full work available at URL: https://arxiv.org/abs/1508.01132

Recommendations

• Transport map accelerated Markov chain Monte Carlo
• Ensemble samplers with affine invariance
• Layered adaptive importance sampling
• Ensemble preconditioning for Markov chain Monte Carlo simulation
• Adaptive multiple importance sampling

zbMATH Keywords

BayesianensembleMCMCimportance samplingresamplinginverse problems

Mathematics Subject Classification ID

Bayesian inference (62F15) Monte Carlo methods (65C05) Numerical analysis or methods applied to Markov chains (65C40)

Cites Work

• Monte Carlo strategies in scientific computing.
• Title not available (Why is that?)
• A nonparametric ensemble transform method for Bayesian inference
• Monte Carlo sampling methods using Markov chains and their applications
• Optimal scaling for various Metropolis-Hastings algorithms.
• Optimal Scaling of Discrete Approximations to Langevin Diffusions
• Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
• On the ergodicity properties of some adaptive MCMC algorithms
• Deterministic Nonperiodic Flow
• Riemann Manifold Langevin and Hamiltonian Monte Carlo Methods
• Adaptive multiple importance sampling
• Bayesian inference with optimal maps
• Bayesian thinking, modeling and computation.
• MCMC methods for functions: modifying old algorithms to make them faster
• Non-reversible Metropolis-Hastings
• Componentwise adaptation for high dimensional MCMC
• Solving large-scale PDE-constrained Bayesian inverse problems with Riemann manifold Hamiltonian Monte Carlo
• The Multiple-Try Method and Local Optimization in Metropolis Sampling
• Ensemble filter techniques for intermittent data assimilation - a survey
• Convergence of adaptive mixtures of importance sampling schemes
• Iterated importance sampling in missing data problems
• Minimum variance importance samplingviaPopulation Monte Carlo
• Efficient MCMC for climate model parameter estimation: parallel adaptive chains and early rejection
• Layered adaptive importance sampling
• Transport Map Accelerated Markov Chain Monte Carlo
• An Adaptive Population Importance Sampler: Learning From Uncertainty

Cited In (1)

• Transport Map Accelerated Adaptive Importance Sampling, and Application to Inverse Problems Arising from Multiscale Stochastic Reaction Networks

This page was built for publication: Ensemble transport adaptive importance sampling

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5228364)