Free energy methods for Bayesian inference: efficient exploration of univariate Gaussian mixture posteriors
From MaRDI portal
Publication:693322
DOI10.1007/S11222-011-9257-9zbMATH Open1252.62015arXiv1003.0428OpenAlexW2013957760MaRDI QIDQ693322FDOQ693322
Gabriel Stoltz, Tony Lelièvre, Nicolas Chopin
Publication date: 7 December 2012
Published in: Statistics and Computing (Search for Journal in Brave)
Abstract: Because of their multimodality, mixture posterior distributions are difficult to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a strategy to enhance the sampling of MCMC in this context, using a biasing procedure which originates from computational Statistical Physics. The principle is first to choose a "reaction coordinate", that is, a "direction" in which the target distribution is multimodal. In a second step, the marginal log-density of the reaction coordinate with respect to the posterior distribution is estimated; minus this quantity is called "free energy" in the computational Statistical Physics literature. To this end, we use adaptive biasing Markov chain algorithms which adapt their targeted invariant distribution on the fly, in order to overcome sampling barriers along the chosen reaction coordinate. Finally, we perform an importance sampling step in order to remove the bias and recover the true posterior. The efficiency factor of the importance sampling step can easily be estimated emph{a priori} once the bias is known, and appears to be rather large for the test cases we considered. A crucial point is the choice of the reaction coordinate. One standard choice (used for example in the classical Wang-Landau algorithm) is minus the log-posterior density. We discuss other choices. We show in particular that the hyper-parameter that determines the order of magnitude of the variance of each component is both a convenient and an efficient reaction coordinate. We also show how to adapt the method to compute the evidence (marginal likelihood) of a mixture model. We illustrate our approach by analyzing two real data sets.
Full work available at URL: https://arxiv.org/abs/1003.0428
Recommendations
- Comparing two Bayes methods based on the free energy functions in Bernoulli mixtures
- Variational Bayesian inference with Gaussian-mixture approximations
- An alternative view of variational Bayes and asymptotic approximations of free energy
- Variational Bayesian learning for parameter estimations of mixture of Gaussians
- Variational approximations in Bayesian model selection for finite mixture distributions
- Inferencia bayesiana en mixturas: metodos aproximados
mixture modelsimportance samplingadaptive Markov chain Monte Carloadaptive biasing potentialadaptive biasing force
Cites Work
- Statistical analysis of finite mixture distributions
- Finite mixture models
- Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling
- Finite mixture and Markov switching models.
- Title not available (Why is that?)
- Title not available (Why is that?)
- Computational and Inferential Difficulties with Mixture Posterior Distributions
- Markov chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models
- Title not available (Why is that?)
- Title not available (Why is that?)
- Sequential Imputations and Bayesian Missing Data Problems
- Monte Carlo sampling methods using Markov chains and their applications
- Simulating normalizing constants: From importance sampling to bridge sampling to path sampling
- Title not available (Why is that?)
- Free energy computations. A mathematical perspective
- The Wang-Landau algorithm in general state spaces: applications and convergence analysis
- Long-time convergence of an adaptive biasing force method
- A Generalized Wang–Landau Algorithm for Monte Carlo Computation
- Computation of free energy differences through nonequilibrium stochastic dynamics: The reaction coordinate case
- Trajectory averaging for stochastic approximation MCMC algorithms
- From microphysics to macrophysics. Methods and applications of statistical physics. II. Translated from the French by D. ter Haar.
- Existence, uniqueness and convergence of a particle approximation for the Adaptive Biasing Force process
- Long-time convergence of an adaptive biasing force method: the bi-channel case
Cited In (21)
- Langevin dynamics with constraints and computation of free energy differences
- On‐line partitioning of the sample space in the regional adaptive algorithm
- Ensemble Markov Chain Monte Carlo with Teleporting Walkers
- Bayesian methods of representative values of variable actions
- Sparse variational Bayesian approximations for nonlinear inverse problems: applications in nonlinear elastography
- Free energy computations by minimization of Kullback-Leibler divergence: An efficient adaptive biasing potential method for sparse representations
- Local and global perspectives on diffusion maps in the analysis of molecular systems
- Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion
- Stratification as a General Variance Reduction Method for Markov Chain Monte Carlo
- A surrogate accelerated multicanonical Monte Carlo method for uncertainty quantification
- Convergence of the Wang-Landau algorithm
- Convergence and efficiency of adaptive importance sampling techniques with partial biasing
- Multicanonical MCMC for sampling rare events: an illustrative review
- Efficient Numerical Algorithms for the Generalized Langevin Equation
- Reconstructing the energy landscape of a distribution from Monte Carlo samples
- Stochastic approximation Monte Carlo importance sampling for approximating exact conditional probabilities
- Partial differential equations and stochastic methods in molecular dynamics
- Ensemble preconditioning for Markov chain Monte Carlo simulation
- Self-healing umbrella sampling: convergence and efficiency
- Multimodal, high-dimensional, model-based, Bayesian inverse problems with applications in biomechanics
- Analysis of an adaptive biasing force method based on self-interacting dynamics
This page was built for publication: Free energy methods for Bayesian inference: efficient exploration of univariate Gaussian mixture posteriors
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q693322)
