Markov chain approximation for Hamilton-Jacobi-Bellman equation with absorbing boundary (Q6490239)

The author investigates the infinite horizon optimal control problem of the controlled density-dependent Markov process (DDMP). He studies the relationship between two value functions for the Markov process and its limit process by using the Hamilton-Jacobi-Bellman (HJB) equation and the viscosity solution method. The author characterizes the value function for controlled DDMP as the solution of some difference type HJB equation. Then he proves that the value function of the controlled DDMP converges to one of its limit processes which is the viscosity solution of the associated HJB equation. An example with the comparison between the explicit solutions of the HJB equations for the DDMP and its fluid limit is finally presented.