Irreversible Langevin samplers and variance reduction: a large deviations approach

DOI10.1088/0951-7715/28/7/2081zbMATH Open1338.60086arXiv1404.0105OpenAlexW3102325779WikidataQ57453404 ScholiaQ57453404MaRDI QIDQ2944157FDOQ2944157

Authors: Luc Rey-Bellet, Konstantinos Spiliopoulos 

Publication date: 28 August 2015

Published in: Nonlinearity (Search for Journal in Brave)

Abstract: In order to sample from a given target distribution (often of Gibbs type), the Monte Carlo Markov chain method consists in constructing an ergodic Markov process whose invariant measure is the target distribution. By sampling the Markov process one can then compute, approximately, expectations of observables with respect to the target distribution. Often the Markov processes used in practice are time-reversible (i.e., they satisfy detailed balance), but our main goal here is to assess and quantify how the addition of a non-reversible part to the process can be used to improve the sampling properties. We focus on the diffusion setting (overdamped Langevin equations) where the drift consists of a gradient vector field as well as another drift which breaks the reversibility of the process but is chosen to preserve the Gibbs measure. In this paper we use the large deviation rate function for the empirical measure as a tool to analyze the speed of convergence to the invariant measure. We show that the addition of an irreversible drift leads to a larger rate function and it strictly improves the speed of convergence of ergodic average for (generic smooth) observables. We also deduce from this result that the asymptotic variance decreases under the addition of the irreversible drift and we give an explicit characterization of the observables whose variance is not reduced reduced, in terms of a nonlinear Poisson equation. Our theoretical results are illustrated and supplemented by numerical simulations.

Full work available at URL: https://arxiv.org/abs/1404.0105

Recommendations

• Variance reduction for irreversible Langevin samplers and diffusion on graphs
• Variance reduction using nonreversible Langevin samplers
• Improving the convergence of reversible samplers
• Variance reduction for diffusions
• Markov chain Monte Carlo and irreversibility

zbMATH Keywords

variance reductionlarge deviationsLangevin equationnon-reversibilityMonte Carlo Markov chain method

Mathematics Subject Classification ID

Computational methods in Markov chains (60J22) Monte Carlo methods (65C05) Numerical analysis or methods applied to Markov chains (65C40) Large deviations (60F10) Central limit and other weak theorems (60F05) Markov chains (discrete-time Markov processes on discrete state spaces) (60J10) Diffusion processes (60J60) Continuous-time Markov processes on general state spaces (60J25)

Cited In (39)

• Optimal variance reduction for Markov chain Monte Carlo
• Generalizing Parallel Replica Dynamics: Trajectory Fragments, Asynchronous Computing, and PDMPs
• Analysis of multiscale integrators for multiple attractors and irreversible Langevin samplers
• Equilibrium large deviations for mean-field systems with translation invariance
• Measuring sample quality with diffusions
• Reversible and non-reversible Markov chain Monte Carlo algorithms for reservoir simulation problems
• Variance reduction using nonreversible Langevin samplers
• Non-reversible Metropolis-Hastings
• Large deviations of empirical measures of diffusions in weighted topologies
• Variance reduction for diffusions
• Using perturbed underdamped Langevin dynamics to efficiently sample from probability distributions
• Large deviations for the empirical measure of a diffusion via weak convergence methods
• Non-reversible metastable diffusions with Gibbs invariant measure. I: Eyring-Kramers formula
• Large deviations for the skew-detailed-balance lifted-Markov processes to sample the equilibrium distribution of the Curie–Weiss model
• Long-time behaviour of degenerate diffusions: UFG-type SDEs and time-inhomogeneous hypoelliptic processes
• Improving the convergence of reversible samplers
• Partial differential equations and stochastic methods in molecular dynamics
• Acceleration of convergence to equilibrium in Markov chains by breaking detailed balance
• Irreversible samplers from jump and continuous Markov processes
• On Irreversible Metropolis Sampling Related to Langevin Dynamics
• Strong invariance principles for ergodic Markov processes
• Extending the Regime of Linear Response with Synthetic Forcings
• Non-reversible metastable diffusions with Gibbs invariant measure. II: Markov chain convergence
• Posterior computation with the Gibbs zig-zag sampler
• The forward-backward envelope for sampling with the overdamped Langevin algorithm
• Large deviations for the empirical measure of the zig-zag process
• Variance reduction for irreversible Langevin samplers and diffusion on graphs
• Quantification of model uncertainty on path-space via goal-oriented relative entropy
• Variational principles for asymptotic variance of general Markov processes
• Geometry-informed irreversible perturbations for accelerated convergence of Langevin dynamics
• Ergodicity of the zigzag process
• Markov chain Monte Carlo and irreversibility
• Optimal scaling of the MALA algorithm with irreversible proposals for Gaussian targets
• Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach
• Ensemble preconditioning for Markov chain Monte Carlo simulation
• Accelerated convergence to equilibrium and reduced asymptotic variance for Langevin dynamics using Stratonovich perturbations
• A large deviation principle for the empirical measures of Metropolis-Hastings chains
• The entropy production of stationary diffusions
• Limit theorems for the zig-zag process

This page was built for publication: Irreversible Langevin samplers and variance reduction: a large deviations approach

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2944157)