Reinforced optimal control

DOI10.4310/CMS.2022.V20.N7.A7zbMATH Open1503.93050arXiv2011.12382MaRDI QIDQ2103076FDOQ2103076

Authors: Christian Bayer, D. Belomestny, Paul Hager, Paolo Pigato, John Schoenmakers, V. G. Spokoiny 

Publication date: 13 December 2022

Published in: Communications in Mathematical Sciences (Search for Journal in Brave)

Abstract: Least squares Monte Carlo methods are a popular numerical approximation method for solving stochastic control problems. Based on dynamic programming, their key feature is the approximation of the conditional expectation of future rewards by linear least squares regression. Hence, the choice of basis functions is crucial for the accuracy of the method. Earlier work by some of us [Belomestny, Schoenmakers, Spokoiny, Zharkynbay. Commun.~Math.~Sci., 18(1):109-121, 2020](arXiv:1808.02341) proposes to reinforce the basis functions in the case of optimal stopping problems by already computed value functions for later times, thereby considerably improving the accuracy with limited additional computational cost. We extend the reinforced regression method to a general class of stochastic control problems, while considerably improving the method's efficiency, as demonstrated by substantial numerical examples as well as theoretical analysis.

Full work available at URL: https://arxiv.org/abs/2011.12382

Recommendations

• Optimal stopping via reinforced regression
• An efficient algorithm for stochastic optimal control problems by means of a least-squares Monte-Carlo method
• Regression methods for stochastic control problems and their convergence analysis
• Generic improvements to least squares Monte Carlo methods with applications to optimal stopping problems
• Monte Carlo methods via a dual approach for some discrete time stochastic control problems

zbMATH Keywords

Monte Carloregressionoptimal controlreinforcement learning

Mathematics Subject Classification ID

Dynamic programming in optimal control and differential games (49L20) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20) Least squares and related methods for stochastic control systems (93E24)

Cited In (2)

• Regression methods for stochastic control problems and their convergence analysis
• Premium control with reinforcement learning

This page was built for publication: Reinforced optimal control

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2103076)