Maximal mean/standard deviation ratio in an undiscounted MDP (Q1064974)

A stationary policy in an MDP (Markov decision process) induces a stationary probability distribution of the reward from each initial state. The problem analyzed here is maximization of the mean/standard deviation ratio of the stationary distribution. In the unichain case, a solution is obtained via parametric analysis of a linear program having the same number of variables and one more constraint than the formulation for gain-rate optimization. The same linear program suffices in the multichain case if the initial state is an element of choice. The easier problem of maximizing the mean/variance ratio is mentioned at the end of the paper.