Markov decision processes on Borel spaces with total cost and random horizon (Q467433): Difference between revisions

The paper deals with Markov decision processes (MDPs) on Borel spaces with possibly unbounded costs. It was motivated by the study of the discounted optimal control problem given in the book by \textit{M. L. Puterman} [Markov decision processes: discrete stochastic dynamic programming. New York, NY: John Wiley \& Sons (1994; Zbl 0829.90134)]. In the book it is proved that the discounted control problem can be treated as a control problem where the horizon is a random variable, which is supposed to follow a geometric distribution independent of the process. The results of the paper are obtained with the help of a dynamic programming approach. They permit working with discounted control problem with varying-time discount factor, possibly depending on the state of the system and the corresponding action as well. To illustrate the theory developed, a version of the linear-quadratic model with a random horizon and a logarithm consumption-investment model are presented.