Nonlinear Monte Carlo methods with polynomial runtime for Bellman equations of discrete time high-dimensional stochastic optimal control problems
From MaRDI portal
Publication:6428573
arXiv2303.03390MaRDI QIDQ6428573FDOQ6428573
Authors: Christian Beck, Arnulf Jentzen, Konrad Kleinberg, Thomas Kruse 
Publication date: 3 March 2023
Abstract: Discrete time stochastic optimal control problems and Markov decision processes (MDPs), respectively, serve as fundamental models for problems that involve sequential decision making under uncertainty and as such constitute the theoretical foundation of reinforcement learning. In this article we study the numerical approximation of MDPs with infinite time horizon, finite control set, and general state spaces. Our set-up in particular covers infinite-horizon optimal stopping problems of discrete time Markov processes. A key tool to solve MDPs are Bellman equations which characterize the value functions of the MDPs and determine the optimal control strategies. By combining ideas from the full-history recursive multilevel Picard approximation method, which was recently introduced to solve certain nonlinear partial differential equations, and ideas from -learning we introduce a class of suitable nonlinear Monte Carlo methods and prove that the proposed methods do overcome the curse of dimensionality in the numerical approximation of the solutions of Bellman equations and the associated discrete time stochastic optimal control problems.
Monte Carlo methods (65C05) Dynamic programming (90C39) Discrete-time Markov processes on general state spaces (60J05) Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)
This page was built for publication: Nonlinear Monte Carlo methods with polynomial runtime for Bellman equations of discrete time high-dimensional stochastic optimal control problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6428573)
