State-Dependent Temperature Control for Langevin Diffusions

DOI10.1137/21M1429424zbMATH Open1493.93057arXiv2011.07456MaRDI QIDQ5080490FDOQ5080490

Authors: Xuefeng Gao, Zuo Quan Xu, Xun Yu Zhou 

Publication date: 31 May 2022

Published in: SIAM Journal on Control and Optimization (Search for Journal in Brave)

Abstract: We study the temperature control problem for Langevin diffusions in the context of non-convex optimization. The classical optimal control of such a problem is of the bang-bang type, which is overly sensitive to errors. A remedy is to allow the diffusions to explore other temperature values and hence smooth out the bang-bang control. We accomplish this by a stochastic relaxed control formulation incorporating randomization of the temperature control and regularizing its entropy. We derive a state-dependent, truncated exponential distribution, which can be used to sample temperatures in a Langevin algorithm, in terms of the solution to an HJB partial differential equation. We carry out a numerical experiment on a one-dimensional baseline example, in which the HJB equation can be easily solved, to compare the performance of the algorithm with three other available algorithms in search of a global optimum.

Full work available at URL: https://arxiv.org/abs/2011.07456

Recommendations

• scientific article; zbMATH DE number 802649
• Langevin thermostat for robust configurational and kinetic sampling
• Control of Quantum Langevin Equations
• Stochastic control and nonequilibrium thermodynamical systems
• Stochastic Control and Nonequilibrium Thermodynamics: Fundamental Limits
• scientific article
• Dissipation and control in microscopic nonequilibrium systems
• Controlled diffusions in a random medium
• scientific article; zbMATH DE number 1071411
• On the use of stochastic differential geometry for non-equilibrium thermodynamic modeling and control

zbMATH Keywords

nonconvex optimizationHJB equationLangevin diffusionentropy regularizationstochastic relaxed controlBoltzmann exploration

Mathematics Subject Classification ID

Nonconvex programming, global optimization (90C26) Existence of optimal solutions belonging to restricted classes (Lipschitz controls, bang-bang controls, etc.) (49J30) Optimal stochastic control (93E20)

Cites Work

• Optimization by simulated annealing
• Title not available (Why is that?)
• Title not available (Why is that?)
• Metastability in reversible diffusion processes. I: Sharp asymptotics for capacities and exit times
• Metastability in reversible diffusion processes. II: Precise asymptotics for small eigenvalues
• Reinforcement learning. An introduction
• Ergodicity for SDEs and approximations: locally Lipschitz vector fields and degenerate noise.
• Recursive Stochastic Algorithms for Global Optimization in $\mathbb{R}^d $
• Smooth Regularization of Bang-Bang Optimal Control Problems
• New smoothing techniques for solving bang–bang optimal control problems—numerical results and statistical interpretation
• Diffusions for Global Optimization
• Diffusion processes with continuous coefficients, I
• Asymptotics of the spectral gap with applications to the theory of simulated annealing
• Diffusion for Global Optimization in $\mathbb{R}^n $
• Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
• Solving high-dimensional partial differential equations using deep learning
• An improved annealing method and its large-time behavior
• Convergence rates for annealing diffusion processes
• Title not available (Why is that?)
• On stationary-point hitting time and ergodicity of stochastic gradient Langevin dynamics
• Weight-preserving simulated tempering

Cited In (9)

• Approximate Optimal Controls via Instanton Expansion for Low Temperature Free Energy Computation
• Exploratory HJB equations and their convergence
• Convergence of simulated annealing using kinetic Langevin dynamics
• Regular and exploratory resource extraction models considering sustainability
• Choquet Regularization for Continuous-Time Reinforcement Learning
• Exploratory Control with Tsallis Entropy for Latent Factor Models
• Stochastic gradient descent and fast relaxation to thermodynamic equilibrium: a stochastic control approach
• Numerical analysis of an extended mean field game for harvesting common fishery resource
• Tail probability estimates of continuous-time simulated annealing processes

This page was built for publication: State-Dependent Temperature Control for Langevin Diffusions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5080490)