Controlled interacting particle algorithms for simulation-based reinforcement learning
From MaRDI portal
Publication:2107628
DOI10.1016/J.SYSCONLE.2022.105392zbMATH Open1505.49027arXiv2107.01244OpenAlexW4309220322MaRDI QIDQ2107628FDOQ2107628
Amirhossein Taghvaei, Anant A. Joshi, Prashant G. Mehta, Sean P. Meyn
Publication date: 2 December 2022
Published in: Systems \& Control Letters (Search for Journal in Brave)
Abstract: This paper is concerned with optimal control problems for control systems in continuous time, and interacting particle system methods designed to construct approximate control solutions. Particular attention is given to the linear quadratic (LQ) control problem. There is a growing interest in re-visiting this classical problem, in part due to the successes of reinforcement learning (RL). The main question of this body of research (and also of our paper) is to approximate the optimal control law {em without} explicitly solving the Riccati equation. A novel simulation-based algorithm, namely a dual ensemble Kalman filter (EnKF), is introduced. The algorithm is used to obtain formulae for optimal control, expressed entirely in terms of the EnKF particles. An extension to the nonlinear case is also presented. The theoretical results and algorithms are illustrated with numerical experiments.
Full work available at URL: https://arxiv.org/abs/2107.01244
Recommendations
- Convergence results for an averaged LQR problem with applications to reinforcement learning
- Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach
- Optimal control for unknown mean-field discrete-time system based on Q-learning
- Reinforcement learning-based direct adaptive optimal control of JLQ model
- Stochastic linear quadratic optimal control for continuous-time systems based on policy iteration
Cites Work
- Title not available (Why is that?)
- Title not available (Why is that?)
- Title not available (Why is that?)
- Adapted solution of a backward stochastic differential equation
- Introduction to stochastic control theory
- Data Assimilation
- A survey of numerical methods for stochastic differential equations
- Stochastic calculus with anticipating integrands
- Backward stochastic differential equations and applications to optimal control
- Adaptive importance sampling for control and inference
- Optimal Control and Nonlinear Filtering for Nondegenerate Diffusion Processes
- Title not available (Why is that?)
- Maximum-likelihood recursive nonlinear filtering
- Multivariable feedback particle filter
- A Variational Approach to Nonlinear Estimation
- Poisson's Equation in Nonlinear Filtering
- A dynamical systems framework for intermittent data assimilation
- Feedback Particle Filter
- Exit probabilities and optimal stochastic control
- Finite Dimensional Linear Systems
- Approximate McKean–Vlasov representations for a class of SPDEs
- Path integrals and symmetry breaking for optimal control theory
- Asymptotic Stability of the Optimal Filter with Respect to Its Initial Condition
- On the stability of matrix-valued Riccati diffusions
- An Optimal Control Derivation of Nonlinear Smoothing Equations
- On the Stability of Kalman--Bucy Diffusion Processes
- Probabilistic Forecasting and Bayesian Data Assimilation
- On the sample complexity of the linear quadratic regulator
- How to Avoid the Curse of Dimensionality: Scalability of Particle Filters with and without Importance Weights
- Diffusion Map-based Algorithm for Gain Function Approximation in the Feedback Particle Filter
- McKean--Vlasov SDEs in Nonlinear Filtering
- Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systems
- Convergence and Sample Complexity of Gradient Methods for the Model-Free Linear–Quadratic Regulator Problem
- Optimal Transportation Methods in Nonlinear Filtering
Cited In (1)
Uses Software
This page was built for publication: Controlled interacting particle algorithms for simulation-based reinforcement learning
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2107628)
