Controlled interacting particle algorithms for simulation-based reinforcement learning
From MaRDI portal
Publication:2107628
Abstract: This paper is concerned with optimal control problems for control systems in continuous time, and interacting particle system methods designed to construct approximate control solutions. Particular attention is given to the linear quadratic (LQ) control problem. There is a growing interest in re-visiting this classical problem, in part due to the successes of reinforcement learning (RL). The main question of this body of research (and also of our paper) is to approximate the optimal control law {em without} explicitly solving the Riccati equation. A novel simulation-based algorithm, namely a dual ensemble Kalman filter (EnKF), is introduced. The algorithm is used to obtain formulae for optimal control, expressed entirely in terms of the EnKF particles. An extension to the nonlinear case is also presented. The theoretical results and algorithms are illustrated with numerical experiments.
Recommendations
- Convergence results for an averaged LQR problem with applications to reinforcement learning
- Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach
- Optimal control for unknown mean-field discrete-time system based on Q-learning
- Reinforcement learning-based direct adaptive optimal control of JLQ model
- Stochastic linear quadratic optimal control for continuous-time systems based on policy iteration
Cites work
- scientific article; zbMATH DE number 3181381 (Why is no real title available?)
- scientific article; zbMATH DE number 1478492 (Why is no real title available?)
- scientific article; zbMATH DE number 802915 (Why is no real title available?)
- A Variational Approach to Nonlinear Estimation
- A dynamical systems framework for intermittent data assimilation
- A survey of numerical methods for stochastic differential equations
- Adapted solution of a backward stochastic differential equation
- Adaptive importance sampling for control and inference
- An optimal control derivation of nonlinear smoothing equations
- Approximate McKean-Vlasov representations for a class of SPDEs
- Asymptotic Stability of the Optimal Filter with Respect to Its Initial Condition
- Backward stochastic differential equations and applications to optimal control
- Convergence and Sample Complexity of Gradient Methods for the Model-Free Linear–Quadratic Regulator Problem
- Data Assimilation
- Derivative-free methods for policy optimization: guarantees for linear quadratic systems
- Diffusion map-based algorithm for gain function approximation in the feedback particle filter
- Exit probabilities and optimal stochastic control
- Feedback Particle Filter
- Finite Dimensional Linear Systems
- How to avoid the curse of dimensionality: scalability of particle filters with and without importance weights
- Introduction to stochastic control theory
- Maximum-likelihood recursive nonlinear filtering
- McKean--Vlasov SDEs in Nonlinear Filtering
- Multivariable feedback particle filter
- On the sample complexity of the linear quadratic regulator
- On the stability of Kalman-Bucy diffusion processes
- On the stability of matrix-valued Riccati diffusions
- Optimal Control and Nonlinear Filtering for Nondegenerate Diffusion Processes
- Optimal Transportation Methods in Nonlinear Filtering
- Path integrals and symmetry breaking for optimal control theory
- Poisson's equation in nonlinear filtering
- Probabilistic Forecasting and Bayesian Data Assimilation
- Reinforcement learning. An introduction
- Stochastic calculus with anticipating integrands
Cited in
(2)
This page was built for publication: Controlled interacting particle algorithms for simulation-based reinforcement learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2107628)
