Average-Cost Optimality Results for Borel-Space Markov Decision Processes with Universally Measurable Policies

arXiv2104.00181MaRDI QIDQ6364294FDOQ6364294

Authors: Huizhen Yu 

Publication date: 31 March 2021

Abstract: We consider discrete-time Markov Decision Processes with Borel state and action spaces and universally measurable policies. For several long-run average cost criteria, we establish the following optimality results: the optimal average cost functions are lower semianalytic, there exist universally measurable semi-Markov or history-dependent $epsilon$-optimal policies, and similar results hold for the minimum average costs achievable by Markov or stationary policies. We then analyze the structure of the optimal average cost functions, proving sufficient conditions for them to be constant almost everywhere with respect to certain $sigma$-finite measures. The most important condition here is that each subset of states with positive measure be reachable with probability one under some policy. We obtain our results by exploiting an inequality for the optimal average cost functions and its connection with submartingales, and, in a special case that involves stationary policies, also by using the theory of recurrent Markov chains.

Mathematics Subject Classification ID

Markov and semi-Markov decision processes (90C40) Optimal stochastic control (93E20)

This page was built for publication: Average-Cost Optimality Results for Borel-Space Markov Decision Processes with Universally Measurable Policies

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6364294)