Consistency of a simple multidimensional scheme for Hamilton-Jacobi-Bellman equations (Q1773340)

The authors consider a multi-valued controlled Markov diffusion on \(X=\mathbb R^n\) \((n\geq 1)\), given as \[ x(t)=x + \int_0^tf(x(s),u(s))ds + \int_0^t\sigma (X(s),u(s))\,dW_s, \] where \((W_t)_{0\leq t\leq \infty}\) is a standard Brownian motion. The control \(u\) is a Lebesgue measurable function with values in a compact set. Further, they assume an infinite horizon discounted payoff and the value function as the minimum value of the payoff, then they consider the associated Hamilton-Jacobi-Bellman (HJB) equation. The authors propose an algorithm for the HJB equation, a Markov decision process approximation, using discretised time steps and diffusion steps. Its consistency is proved, which in turn implies its convergence. The algorithm has the advantage of being easy to implement in any dimension.