On Markovian decision programming with recursive reward functions (Q2640462)

The paper deals with a discrete time Markov decision model with infinite horizon and recursive reward functions. This model was considered earlier by \textit{N. Furukawa} and \textit{S. Iwamoto} [Bull. Math. Statist. 15, No.3/4, 79-91 (1973; Zbl 0304.90117)]. The state and action sets are countable or finite. The authors establish the optimality principle and optimality equation when the model satisfies some additional conditions. They prove the existence of Markov optimal and \(\epsilon\)-optimal policies for nonhomogeneous models. For homogeneous models the existence of stationary optimal and \(\epsilon\)-optimal policies is proved. The authors also consider a policy iteration algorithm for homogeneous models.