Convergence rates of Markov chain approximation methods for controlled diffusions with stopping (Q601074)

The numerical methods using Markov chain approximations are considered for a one-dimensional controlled diffusion process \[ X^{x,u}(t)=x+\int_0^t b(X^{x,u}(s), u(s))ds +\int_0^t\sigma(X^{x,u}(s))dW(s) \] with initial data \(x\), given control \(u=u(t)\) and a standard real-valued Wiener process \(W(t)\). For \(B>0\) the first exit time from \((-B,B)\), i.e., \(\tau_B^{x,u}=\inf \{t:X^{x,u}(t) \notin (-B,B)\}\) is studied. The value function describes the optimal first exit time \(V(x,B)=\inf_{u}\tau_B^{x,u}\). The explicit representation of the dependence of \(V(x,B)\) on \(B\) is a novelty of the analysis of the problem. The considered numerical methods use a purely probabilistic approach. To prove the convergence of the algorithm by a Markov chain approximation method, a tangency problem might arise due to discontinuity of the first exit time with respect to discretized \(B\). In the considered setup, it is demonstrated that the tangency problem will not arise in the sense of convergence in probability and in \(L^1\), the analysis of the dependence of \(V(x,B)\) on \(B\) is used. In addition, controlled diffusions with a discount factor are considered.