Free energy methods for Bayesian inference: efficient exploration of univariate Gaussian mixture posteriors
From MaRDI portal
Publication:693322
Abstract: Because of their multimodality, mixture posterior distributions are difficult to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a strategy to enhance the sampling of MCMC in this context, using a biasing procedure which originates from computational Statistical Physics. The principle is first to choose a "reaction coordinate", that is, a "direction" in which the target distribution is multimodal. In a second step, the marginal log-density of the reaction coordinate with respect to the posterior distribution is estimated; minus this quantity is called "free energy" in the computational Statistical Physics literature. To this end, we use adaptive biasing Markov chain algorithms which adapt their targeted invariant distribution on the fly, in order to overcome sampling barriers along the chosen reaction coordinate. Finally, we perform an importance sampling step in order to remove the bias and recover the true posterior. The efficiency factor of the importance sampling step can easily be estimated emph{a priori} once the bias is known, and appears to be rather large for the test cases we considered. A crucial point is the choice of the reaction coordinate. One standard choice (used for example in the classical Wang-Landau algorithm) is minus the log-posterior density. We discuss other choices. We show in particular that the hyper-parameter that determines the order of magnitude of the variance of each component is both a convenient and an efficient reaction coordinate. We also show how to adapt the method to compute the evidence (marginal likelihood) of a mixture model. We illustrate our approach by analyzing two real data sets.
Recommendations
- Comparing two Bayes methods based on the free energy functions in Bernoulli mixtures
- Variational Bayesian inference with Gaussian-mixture approximations
- An alternative view of variational Bayes and asymptotic approximations of free energy
- Variational Bayesian learning for parameter estimations of mixture of Gaussians
- Variational approximations in Bayesian model selection for finite mixture distributions
- Inferencia bayesiana en mixturas: metodos aproximados
Cites work
- scientific article; zbMATH DE number 410743 (Why is no real title available?)
- scientific article; zbMATH DE number 5133222 (Why is no real title available?)
- scientific article; zbMATH DE number 597901 (Why is no real title available?)
- scientific article; zbMATH DE number 1085980 (Why is no real title available?)
- scientific article; zbMATH DE number 1448982 (Why is no real title available?)
- A Generalized Wang–Landau Algorithm for Monte Carlo Computation
- Computation of free energy differences through nonequilibrium stochastic dynamics: The reaction coordinate case
- Computational and Inferential Difficulties with Mixture Posterior Distributions
- Existence, uniqueness and convergence of a particle approximation for the adaptive biasing force process
- Finite mixture and Markov switching models.
- Finite mixture models
- Free energy computations. A mathematical perspective
- From microphysics to macrophysics. Methods and applications of statistical physics. II. Translated from the French by D. ter Haar.
- Long-time convergence of an adaptive biasing force method
- Long-time convergence of an adaptive biasing force method: the bi-channel case
- Markov chain Monte Carlo Estimation of Classical and Dynamic Switching and Mixture Models
- Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling
- Monte Carlo sampling methods using Markov chains and their applications
- Sequential Imputations and Bayesian Missing Data Problems
- Simulating normalizing constants: From importance sampling to bridge sampling to path sampling
- Statistical analysis of finite mixture distributions
- The Wang-Landau algorithm in general state spaces: applications and convergence analysis
- Trajectory averaging for stochastic approximation MCMC algorithms
Cited in
(24)- Analysis of an adaptive biasing force method based on self-interacting dynamics
- Bayesian inference via projections
- Langevin dynamics with constraints and computation of free energy differences
- Comparing two Bayes methods based on the free energy functions in Bernoulli mixtures
- On‐line partitioning of the sample space in the regional adaptive algorithm
- Bayesian methods of representative values of variable actions
- Sparse variational Bayesian approximations for nonlinear inverse problems: applications in nonlinear elastography
- Free energy computations by minimization of Kullback-Leibler divergence: An efficient adaptive biasing potential method for sparse representations
- Local and global perspectives on diffusion maps in the analysis of molecular systems
- Optimal non-reversible linear drift for the convergence to equilibrium of a diffusion
- A surrogate accelerated multicanonical Monte Carlo method for uncertainty quantification
- Stratification as a General Variance Reduction Method for Markov Chain Monte Carlo
- Convergence and efficiency of adaptive importance sampling techniques with partial biasing
- Convergence of the Wang-Landau algorithm
- Multicanonical MCMC for sampling rare events: an illustrative review
- Ensemble Markov chain Monte Carlo with teleporting walkers
- Reconstructing the energy landscape of a distribution from Monte Carlo samples
- Efficient Numerical Algorithms for the Generalized Langevin Equation
- Stochastic approximation Monte Carlo importance sampling for approximating exact conditional probabilities
- Partial differential equations and stochastic methods in molecular dynamics
- Convergence rates for an adaptive biasing potential scheme from a Wasserstein optimization perspective
- Ensemble preconditioning for Markov chain Monte Carlo simulation
- Self-healing umbrella sampling: convergence and efficiency
- Multimodal, high-dimensional, model-based, Bayesian inverse problems with applications in biomechanics
This page was built for publication: Free energy methods for Bayesian inference: efficient exploration of univariate Gaussian mixture posteriors
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q693322)
