Singulary perturbed Markov control problem: Limiting average cost (Q1174700)

The paper deals with a perturbed Markov decision process with discrete time and finite state and action sets. The criterion is average rewards per unit time. The authors assume that the space may be decomposed into several non-overlapping subsets and each of these subsets is an ergodic class for all Markov chains generated by stationary strategies. The perturbation vectors depend on states and actions. It is assumed that, if a perturbation parameter is small, any perturbed Markov chain, generated by a stationary strategy, is irreducible. The authors prove that an optimal solution to the limit perturbed problem, when the perturbation parameter tends to 0, can be approximated by an optimal solution to the perturbed problem when the parameter is small. They formulate a nonlinear program in the space of limit state-action frequencies. The solution of this problem determines an optimal limit strategy for a perturbed process.