Computational aspects in applied stochastic control (Q1342439)

This paper outlines four approaches to solving stochastic control problems. These include dynamic programming direct solution of the Hamilton-Jacobi-Bellman equation; the Markov chain approximation; algorithms based on iterative solutions of well-known (and solvable) problems and finally application of simulation and stochastic approximations. A number of applications such as the linear quadratic control problem (in discrete and continuous time), portfolio selection with (and without) transaction costs, inventory control using impulse control theory and other problems are presented and solved. Further, numerical techniques used for solving stochastic control problems are also applied to specific problems and a comparative analysis with the stochastic programming approach is included. To conclude, we review some expert systems for the solution of stochastic control problems. An extensive list of references is included in the paper.