The Markov chain approximation approach for numerical solution of stochastic control problems: experiences from Merton's problem. (Q1856015)

It is well known that many problems of financial economics lead to the solutions of continuous-time, continuous-state stochastic control problems. Since to solve such problems exactly is very complicated, approximative methods have to be mostly employed. The diffusion continuous-time, continuous-state process is then replaced by a discrete-time, discrete-state Markov chain and, consequently, the original problem is (by this approach) replaced by a Markov chain control problem. The aim of this publication is to compare various versions of the Markov chain approximation approach in the case of Merton's simple consumption/investment problem for which an explicit solution is known. It follows from the paper that the above mentioned approximations are suitable. Of course, the behaviour of the approximation depends on the determination of an upper bound on the state space for the approximating Markov chain (namely the original process is unbounded). Moreover, it follows from the paper that the Richardson extrapolation technique can be successively employed to improve the solution.