Optimal control as a graphical model inference problem
From MaRDI portal
Abstract: We reformulate a class of non-linear stochastic optimal control problems introduced by Todorov (2007) as a Kullback-Leibler (KL) minimization problem. As a result, the optimal control computation reduces to an inference computation and approximate inference methods can be applied to efficiently compute approximate optimal controls. We show how this KL control theory contains the path integral control method as a special case. We provide an example of a block stacking task and a multi-agent cooperative game where we demonstrate how approximate inference can be successfully applied to instances that are too complex for exact computation. We discuss the relation of the KL control approach to other inference approaches to control.
Recommendations
- Graphical model inference in optimal control of stochastic multi-agent systems
- A Bayesian view on motor control and planning
- Adaptive importance sampling for control and inference
- An introduction to stochastic control theory, path integrals and reinforcement learning
- Stochastic optimal control of state constrained systems
Cites work
- Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms
- Dynamic programming and influence diagrams
- Efficient computation of optimal actions
- Graphical model inference in optimal control of stochastic multi-agent systems
- scientific article; zbMATH DE number 1321699 (Why is no real title available?)
- scientific article; zbMATH DE number 4121482 (Why is no real title available?)
- scientific article; zbMATH DE number 870530 (Why is no real title available?)
- LibDAI: a free and open source C++ library for discrete approximate inference in graphical models
- Path integrals and symmetry breaking for optimal control theory
- Policy search for motor primitives in robotics
- Study of the starting pressure gradient in branching network
- Using Expectation-Maximization for Reinforcement Learning
Cited in
(39)- An estimator for the relative entropy rate of path measures for stochastic differential equations
- Planning and navigation as active inference
- Sparse randomized shortest paths routing with Tsallis divergence regularization
- On a probabilistic approach to synthesize control policies from example datasets
- Convergence of value functions for finite horizon Markov decision processes with constraints
- Generalised free energy and active inference
- Optimal design of priors constrained by external predictors
- Design of biased random walks on a graph with application to collaborative recommendation
- Online control of simulated humanoids using particle belief propagation
- An introduction to stochastic control theory, path integrals and reinforcement learning
- A KBRL inference metaheuristic with applications
- Adaptive importance sampling for control and inference
- A cost/speed/reliability tradeoff to erasing
- Nonlinear discrete time optimal control based on fuzzy models
- Action selection in growing state spaces: control of network structure growth
- EP for efficient stochastic control with obstacles
- Efficient computation of optimal actions
- Optimal speech motor control and token-to-token variability: a Bayesian modeling approach
- Systems of Bounded Rational Agents with Information-Theoretic Constraints
- A Bayesian view on motor control and planning
- Graphical model inference in optimal control of stochastic multi-agent systems
- Adaptive smoothing for path integral control
- A minimum free energy model of motor learning
- Variational approach to rare event simulation using least-squares regression
- Data assimilation: the Schrödinger perspective
- Learning effective state-feedback controllers through efficient multilevel importance samplers
- A reward-maximizing spiking neuron as a bounded rational decision maker
- A multilevel approach for stochastic nonlinear optimal control
- Variational Inference for Stochastic Differential Equations
- Kullback–Leibler-Quadratic Optimal Control
- The free energy principle made simpler but not too simple
- Reward Maximization Through Discrete Active Inference
- Nonparametric inference of stochastic differential equations based on the relative entropy rate
- Diffusion Schrödinger bridges for Bayesian computation
- Probabilistic control and majorisation of optimal control
- Approximate constrained stochastic optimal control via parameterized input inference
- Bayesian optimal control for a non-autonomous stochastic discrete time system
- Digital twins: McKean-Pontryagin control for partially observed physical twins
- Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
This page was built for publication: Optimal control as a graphical model inference problem
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q420939)
