Optimal risk probability for first passage models in semi-Markov decision processes (Q2272058)

The authors are concerned with the risk minimization problem in semi-Markov denumerable state decision processes. The criterion to be optimized is the risk probability that a fist passage time to some target set does not exceed a threshold value. They characterize such risk functions and the corresponding optimal value function, and prove that the latter satisfies the optimality equation by using a successive approximation technique. Then some properties of optimal policies are presented and conditions for their existence are given. In addition, a value iteration algortihm and a policy improvement method for obtaining respectively, the optimal value function and optimal policies are developed. Two examples are given to illustrate the value iteration procedure and essential characterization of the risk function.