Necessary and sufficient conditions for a bounded solution to the optimality equation in average reward Markov decision chains (Q1103532)

Consider a discrete time Markov decision process with countable state space S. In addition to the standard assumptions of compact action sets and continuous transition probabilities, suppose that the Markov chain determined by each stationary policy f has a single positive recurrent class R(f), which is entered with probability one and which contains at least one member of a fixed, finite subset G of S. The main theorem gives, under these assumptions, five necessary and sufficient conditions (including a simultaneous Doeblin condition with set G) for the average reward optimality equation to have a bounded measurable solution for an arbitrary bounded measurable reward function. The establishment of necessity is an uncommon feature; sufficient conditions are discussed in \textit{L. C. Thomas} [``Connectedness conditions for denumerable state Markov decision processes'', in: Recent developments in Markov decision processes, R. Hartley, L. C. Thomas, D. J. White (eds.), Academic Press (1980; Zbl 0547.90064)].