Approximate Optimal Controls via Instanton Expansion for Low Temperature Free Energy Computation
From MaRDI portal
Optimality conditions for problems involving relations other than differential equations (49K21) Numerical methods in optimal control (49M99) Variational principles of physics (49S05) Large deviations (60F10) Monte Carlo methods (65C05) Monte Carlo methods applied to problems in statistical mechanics (82M31)
Abstract: The computation of free energies is a common issue in statistical physics. A natural technique to compute such high dimensional integrals is to resort to Monte Carlo simulations. However these techniques generally suffer from a high variance in the low temperature regime, because the expectation is often dominated by high values corresponding to rare system trajectories. A standard way to reduce the variance of the estimator is to modify the drift of the dynamics with a control enhancing the probability of rare event, leading to so-called importance sampling estimators. In theory, the optimal control leads to a zero-variance estimator; it is however defined implicitly and computing it is of the same difficulty as the original problem. We propose here a general strategy to build approximate optimal controls in the small temperature limit for diffusion processes, with the first goal to reduce the variance of free energy Monte Carlo estimators. Our construction builds upon low noise asymptotics by expanding the optimal control around the instanton, which is the path describing most likely fluctuations at low temperature. This technique not only helps reducing variance, but it is also interesting as a theoretical tool since it differs from usual small temperature expansions (WKB ansatz). As a complementary consequence of our expansion, we provide a perturbative formula for computing the free energy in the small temperature regime, which refines the now standard Freidlin--Wentzell asymptotics. We compute this expansion explicitly for lower orders, and explain how our strategy can be extended to an arbitrary order of accuracy. We support our findings with illustrative numerical examples.
Recommendations
- Optimal Control of the Keilson-Storer Master Equation in a Monte Carlo Framework
- Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach
- Optimal low-dimensional dynamical approximations
- A Perturbative Approach to Control Variates in Molecular Dynamics
- Langevin dynamics with constraints and computation of free energy differences
- Efficient numerical evaluation of thermodynamic quantities on infinite (semi-)classical chains
- A gentle stochastic thermostat for molecular dynamics
- The rate-controlled constrained-equilibrium approach to far-from-local-equilibrium thermodynamics
- State-Dependent Temperature Control for Langevin Diffusions
Cites work
- Adaptive importance sampling with forward-backward stochastic differential equations
- Adaptive sampling of large deviations
- Applications of the cross-entropy method to importance sampling and optimal control of diffusions
- Arclength Parametrized Hamilton's Equations for the Calculation of Instantons
- Asymptotic series and exit time probabilities
- Continuous-time stochastic control and optimization with financial applications
- Controlled Markov processes and viscosity solutions
- Counterexamples in importance sampling for large deviations probabilities
- Deep learning-based numerical methods for high-dimensional parabolic partial differential equations and backward stochastic differential equations
- Efficient large deviation estimation based on importance sampling
- Ergodic Properties of Markov Processes
- Gel’fand–Yaglom type equations for calculating fluctuations around instantons in stochastic systems
- Generalisation of the Eyring-Kramers transition rate formula to irreversible diffusion processes
- Go with the winners: a general Monte Carlo strategy
- scientific article; zbMATH DE number 1713116 (Why is no real title available?)
- scientific article; zbMATH DE number 5919872 (Why is no real title available?)
- scientific article; zbMATH DE number 51724 (Why is no real title available?)
- scientific article; zbMATH DE number 54145 (Why is no real title available?)
- scientific article; zbMATH DE number 1239549 (Why is no real title available?)
- scientific article; zbMATH DE number 1005873 (Why is no real title available?)
- scientific article; zbMATH DE number 802915 (Why is no real title available?)
- scientific article; zbMATH DE number 5681750 (Why is no real title available?)
- Instanton based importance sampling for rare events in stochastic pdes
- Introduction to rare event simulation.
- Large deviations techniques and applications.
- Long Term Effects of Small Random Perturbations on Dynamical Systems: Theoretical and Computational Tools
- Minimum action method for the study of rare events
- Model reduction algorithms for optimal control and importance sampling of diffusions
- Numerical computation of rare events via large deviation theory
- On a new class of score functions to estimate tail probabilities of some stochastic processes with adaptive multilevel splitting
- On large deviations theory and asymptotically efficient Monte Carlo estimation
- On the Asymptotics of the Transition Density of Processes with Small Diffusion
- On the Control of an Interacting Particle Estimation of Schrödinger Ground States
- On the stability of nonlinear Feynman-Kac semigroups
- On the weak-noise limit of Fokker-Planck models
- Partial differential equations and stochastic methods in molecular dynamics
- Perturbative calculation of quasi-potential in non-equilibrium diffusions: a mean-field example
- Rare event computation in deterministic chaotic systems using genealogical particle analysis
- Rare event simulation for stochastic dynamics in continuous time
- Rare event simulation of small noise diffusions
- Solving high-dimensional partial differential equations using deep learning
- Splitting for rare event simulation: A large deviation approach to design and analysis
- Stochastic simulation: Algorithms and analysis
- Subsolutions of an Isaacs Equation and Efficient Schemes for Importance Sampling
- The geometric minimum action method: A least action principle on the space of curves
- The instanton method and its numerical implementation in fluid mechanics
- Viscosity Solutions of Hamilton-Jacobi Equations
Cited in
(4)- Path integral derivation and numerical computation of large deviation prefactors for non-equilibrium dynamics through matrix Riccati equations
- Symmetries and zero modes in sample path large deviations
- Stochastic viscosity approximations of Hamilton–Jacobi equations and variance reduction
- Scalable methods for computing sharp extreme event probabilities in infinite-dimensional stochastic systems
This page was built for publication: Approximate Optimal Controls via Instanton Expansion for Low Temperature Free Energy Computation
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5157688)
