Optimal threshold probability in undiscounted Markov decision processes with a target set. (Q1427885)

The paper is about minimizing risk models with a threshold criterion \(P_i^\pi(Z \leq r)\) where \(Z=\sum_{k=1}^{\tau-1} Y_k\) and \(\tau\) is a first passage time to a target set. The problem is modeled as undiscounted Markov decision process with discrete time space and countable state, action, and reward space. The main results are that the optimal value function is a unique solution to an optimality equation and that an optimal right continuous stationary policy exists. Some value iteration methods and a policy space iteration method are presented.