A stochastic maximum principle approach for reinforcement learning with parameterized environment
From MaRDI portal
Abstract: In this work, we introduce a stochastic maximum principle (SMP) approach for solving the reinforcement learning problem with the assumption that the unknowns in the environment can be parameterized based on physics knowledge. For the development of numerical algorithms, we shall apply an effective online parameter estimation method as our exploration technique to estimate the environment parameter during the training procedure, and the exploitation for the optimal policy will be achieved by an efficient backward action learning method for policy improvement under the SMP framework. Numerical experiments will be presented to demonstrate that our SMP approach for reinforcement learning can produce reliable control policy, and the gradient descent type optimization in the SMP solver requires less training episodes compared with the standard dynamic programming principle based methods.
Recommendations
- Value iteration for LQR control of unknown stochastic-parameter linear systems
- A generalized path integral control approach to reinforcement learning
- Reinforcement learning with dynamic convex risk measures
- Connecting stochastic optimal control and reinforcement learning
- Reinforcement learning for exploratory linear-quadratic two-person zero-sum stochastic differential games
Cites work
- \({\mathcal Q}\)-learning
- A direct filter method for parameter estimation
- A first order scheme for backward doubly stochastic differential equations
- A General Stochastic Maximum Principle for Optimal Control Problems
- A numerical scheme for BSDEs
- A random map implementation of implicit filters
- A survey of convergence results on particle filtering methods for practitioners
- An efficient gradient projection method for stochastic optimal control problems
- An efficient numerical algorithm for solving data driven feedback control problems
- Data assimilation of synthetic data as a novel strategy for predicting disease progression in alopecia areata
- Higher-order implicit strong numerical schemes for stochastic differential equations
- scientific article; zbMATH DE number 1325009 (Why is no real title available?)
- scientific article; zbMATH DE number 7307478 (Why is no real title available?)
- New kinds of high-order multistep schemes for coupled forward backward stochastic differential equations
- Particle Markov Chain Monte Carlo Methods
- Reinforcement learning. An introduction
Cited in
(3)- Pontryagin maximum principle for fractional delay differential equations and controlled weakly singular Volterra delay integral equations
- Assimilating partial observation to enhance feedback control of stochastic dynamical systems
- An optimal-order backpropagation algorithm for stochastic neural networks
This page was built for publication: A stochastic maximum principle approach for reinforcement learning with parameterized environment
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6105091)
