Optimal control as a graphical model inference problem
From MaRDI portal
Abstract: We reformulate a class of non-linear stochastic optimal control problems introduced by Todorov (2007) as a Kullback-Leibler (KL) minimization problem. As a result, the optimal control computation reduces to an inference computation and approximate inference methods can be applied to efficiently compute approximate optimal controls. We show how this KL control theory contains the path integral control method as a special case. We provide an example of a block stacking task and a multi-agent cooperative game where we demonstrate how approximate inference can be successfully applied to instances that are too complex for exact computation. We discuss the relation of the KL control approach to other inference approaches to control.
Recommendations
- Graphical model inference in optimal control of stochastic multi-agent systems
- A Bayesian view on motor control and planning
- Adaptive importance sampling for control and inference
- An introduction to stochastic control theory, path integrals and reinforcement learning
- Stochastic optimal control of state constrained systems
Cites work
- scientific article; zbMATH DE number 1321699 (Why is no real title available?)
- scientific article; zbMATH DE number 4121482 (Why is no real title available?)
- scientific article; zbMATH DE number 870530 (Why is no real title available?)
- Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms
- Dynamic programming and influence diagrams
- Efficient computation of optimal actions
- Graphical model inference in optimal control of stochastic multi-agent systems
- LibDAI: a free and open source C++ library for discrete approximate inference in graphical models
- Path integrals and symmetry breaking for optimal control theory
- Policy search for motor primitives in robotics
- Study of the starting pressure gradient in branching network
- Using Expectation-Maximization for Reinforcement Learning
Cited in
(38)- Probabilistic control and majorisation of optimal control
- Convergence of value functions for finite horizon Markov decision processes with constraints
- Nonparametric inference of stochastic differential equations based on the relative entropy rate
- A minimum free energy model of motor learning
- scientific article; zbMATH DE number 7307475 (Why is no real title available?)
- Adaptive importance sampling for control and inference
- Optimal design of priors constrained by external predictors
- Kullback–Leibler-Quadratic Optimal Control
- Sparse randomized shortest paths routing with Tsallis divergence regularization
- A Bayesian view on motor control and planning
- The free energy principle made simpler but not too simple
- Generalised free energy and active inference
- Graphical model inference in optimal control of stochastic multi-agent systems
- Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path space
- Nonlinear discrete time optimal control based on fuzzy models
- Action selection in growing state spaces: control of network structure growth
- Efficient computation of optimal actions
- A multilevel approach for stochastic nonlinear optimal control
- Variational Inference for Stochastic Differential Equations
- Variational approach to rare event simulation using least-squares regression
- Data assimilation: the Schrödinger perspective
- Bayesian optimal control for a non-autonomous stochastic discrete time system
- An estimator for the relative entropy rate of path measures for stochastic differential equations
- Planning and navigation as active inference
- Diffusion Schrödinger bridges for Bayesian computation
- Optimal speech motor control and token-to-token variability: a Bayesian modeling approach
- Learning effective state-feedback controllers through efficient multilevel importance samplers
- On a probabilistic approach to synthesize control policies from example datasets
- EP for efficient stochastic control with obstacles
- Systems of Bounded Rational Agents with Information-Theoretic Constraints
- A cost/speed/reliability tradeoff to erasing
- Reward Maximization Through Discrete Active Inference
- An introduction to stochastic control theory, path integrals and reinforcement learning
- Design of biased random walks on a graph with application to collaborative recommendation
- A reward-maximizing spiking neuron as a bounded rational decision maker
- Approximate constrained stochastic optimal control via parameterized input inference
- A KBRL inference metaheuristic with applications
- Online control of simulated humanoids using particle belief propagation
This page was built for publication: Optimal control as a graphical model inference problem
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q420939)
