Path integrals and symmetry breaking for optimal control theory
From MaRDI portal
Publication:4968820
Abstract: This paper considers linear-quadratic control of a non-linear dynamical system subject to arbitrary cost. I show that for this class of stochastic control problems the non-linear Hamilton-Jacobi-Bellman equation can be transformed into a linear equation. The transformation is similar to the transformation used to relate the classical Hamilton-Jacobi equation to the Schr"odinger equation. As a result of the linearity, the usual backward computation can be replaced by a forward diffusion process, that can be computed by stochastic integration or by the evaluation of a path integral. It is shown, how in the deterministic limit the PMP formalism is recovered. The significance of the path integral approach is that it forms the basis for a number of efficient computational methods, such as MC sampling, the Laplace approximation and the variational approximation. We show the effectiveness of the first two methods in number of examples. Examples are given that show the qualitative difference between stochastic and deterministic control and the occurrence of symmetry breaking as a function of the noise.
Recommendations
- Iterative path integral approach to nonlinear stochastic optimal control under compound Poisson noise
- An introduction to stochastic control theory, path integrals and reinforcement learning
- Linear-quadratic stochastic control problem. III. Nonlinear optimal controllers
- A multilevel approach for stochastic nonlinear optimal control
- A practical variational approach to stochastic optimal control via state moment equations
Cites work
- scientific article; zbMATH DE number 3167340 (Why is no real title available?)
- scientific article; zbMATH DE number 1325009 (Why is no real title available?)
- scientific article; zbMATH DE number 2027759 (Why is no real title available?)
- scientific article; zbMATH DE number 3225499 (Why is no real title available?)
- scientific article; zbMATH DE number 3265711 (Why is no real title available?)
- Exit probabilities and optimal stochastic control
- On the approximation of Feynman-Kac path integrals
- Solving ODEs with MATLAB
Cited in
(30)- An introduction to stochastic control theory, path integrals and reinforcement learning
- scientific article; zbMATH DE number 4002016 (Why is no real title available?)
- Stochastic optimal control as non-equilibrium statistical mechanics: calculus of variations over density and current
- Design of biased random walks on a graph with application to collaborative recommendation
- Controlled interacting particle algorithms for simulation-based reinforcement learning
- Temporal deep unfolding for constrained nonlinear stochastic optimal control
- Optimal lock-down intensity: a stochastic pandemic control approach of path integral
- Whence the expected free energy?
- Optimization of market stochastic dynamics
- Optimal control of probabilistic Boolean control networks: A scalable infinite horizon approach
- Optimal control as a graphical model inference problem
- Disordered high-dimensional optimal control
- Iterative path integral approach to nonlinear stochastic optimal control under compound Poisson noise
- A cost/speed/reliability tradeoff to erasing
- The quantum dark side of the optimal control theory
- Nonparametric inference of stochastic differential equations based on the relative entropy rate
- Action selection in growing state spaces: control of network structure growth
- Particle filters with nudging in multiscale chaotic systems: with application to the Lorenz '96 atmospheric model
- Stochastic nonlinear model predictive control based on Gaussian mixture approximations
- Variational and optimal control representations of conditioned and driven processes
- Linearly solvable stochastic control Lyapunov functions
- A fictitious-play finite-difference method for linearly solvable mean field games
- The free energy principle made simpler but not too simple
- Nonlinear stochastic receding horizon control: stability, robustness and Monte Carlo methods for control approximation
- A transformed path integral approach for solution of the Fokker-Planck equation
- Variational approach to rare event simulation using least-squares regression
- Efficient computation of optimal open-loop controls for stochastic systems
- An estimator for the relative entropy rate of path measures for stochastic differential equations
- A multilevel approach for stochastic nonlinear optimal control
- Stochastic optimal control via forward and backward stochastic differential equations and importance sampling
This page was built for publication: Path integrals and symmetry breaking for optimal control theory
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4968820)
