Path integrals and symmetry breaking for optimal control theory
From MaRDI portal
Publication:4968820
DOI10.1088/1742-5468/2005/11/P11011zbMATH Open1456.49027arXivphysics/0505066OpenAlexW2026659355MaRDI QIDQ4968820FDOQ4968820
Authors: Hilbert J. Kappen 
Publication date: 9 July 2019
Published in: Journal of Statistical Mechanics: Theory and Experiment (Search for Journal in Brave)
Abstract: This paper considers linear-quadratic control of a non-linear dynamical system subject to arbitrary cost. I show that for this class of stochastic control problems the non-linear Hamilton-Jacobi-Bellman equation can be transformed into a linear equation. The transformation is similar to the transformation used to relate the classical Hamilton-Jacobi equation to the Schr"odinger equation. As a result of the linearity, the usual backward computation can be replaced by a forward diffusion process, that can be computed by stochastic integration or by the evaluation of a path integral. It is shown, how in the deterministic limit the PMP formalism is recovered. The significance of the path integral approach is that it forms the basis for a number of efficient computational methods, such as MC sampling, the Laplace approximation and the variational approximation. We show the effectiveness of the first two methods in number of examples. Examples are given that show the qualitative difference between stochastic and deterministic control and the occurrence of symmetry breaking as a function of the noise.
Full work available at URL: https://arxiv.org/abs/physics/0505066
Recommendations
- Iterative path integral approach to nonlinear stochastic optimal control under compound Poisson noise
- An introduction to stochastic control theory, path integrals and reinforcement learning
- Linear-quadratic stochastic control problem. III. Nonlinear optimal controllers
- A multilevel approach for stochastic nonlinear optimal control
- A practical variational approach to stochastic optimal control via state moment equations
Optimality conditions for problems involving randomness (49K45) Dynamic programming in optimal control and differential games (49L20) Linear-quadratic optimal control problems (49N10) Optimal stochastic control (93E20) Equilibrium statistical mechanics (82B99)
Cites Work
Cited In (30)
- Stochastic optimal control as non-equilibrium statistical mechanics: calculus of variations over density and current
- Design of biased random walks on a graph with application to collaborative recommendation
- Temporal deep unfolding for constrained nonlinear stochastic optimal control
- Optimal lock-down intensity: a stochastic pandemic control approach of path integral
- Controlled interacting particle algorithms for simulation-based reinforcement learning
- Whence the expected free energy?
- Optimal control of probabilistic Boolean control networks: A scalable infinite horizon approach
- Optimization of market stochastic dynamics
- Disordered high-dimensional optimal control
- Optimal control as a graphical model inference problem
- Iterative path integral approach to nonlinear stochastic optimal control under compound Poisson noise
- A cost/speed/reliability tradeoff to erasing
- The quantum dark side of the optimal control theory
- Nonparametric inference of stochastic differential equations based on the relative entropy rate
- Action selection in growing state spaces: control of network structure growth
- Particle filters with nudging in multiscale chaotic systems: with application to the Lorenz '96 atmospheric model
- Linearly solvable stochastic control Lyapunov functions
- Stochastic nonlinear model predictive control based on Gaussian mixture approximations
- A fictitious-play finite-difference method for linearly solvable mean field games
- Variational and optimal control representations of conditioned and driven processes
- The free energy principle made simpler but not too simple
- Nonlinear stochastic receding horizon control: stability, robustness and Monte Carlo methods for control approximation
- A transformed path integral approach for solution of the Fokker-Planck equation
- Variational approach to rare event simulation using least-squares regression
- Efficient computation of optimal open-loop controls for stochastic systems
- A multilevel approach for stochastic nonlinear optimal control
- An estimator for the relative entropy rate of path measures for stochastic differential equations
- Stochastic optimal control via forward and backward stochastic differential equations and importance sampling
- Title not available (Why is that?)
- An introduction to stochastic control theory, path integrals and reinforcement learning
Uses Software
This page was built for publication: Path integrals and symmetry breaking for optimal control theory
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4968820)
