Optimality criteria for deterministic discrete-time infinite horizon optimization (Q2569984)

The problem of an optimal sequential decision making over an infinite horizon is complicated since total (even discounted) cost frequently is infinite for many different decision policies. The task is to specify a criterion that is neither too overselective (no optimal policy) nor underselective (most policies are optimal). The authors have suggested the criterion of efficiency, meaning that a solution is efficient if it is optimal to each of the states through which it passes. They have shown that efficient solutions exist under mild conditions and thus the criterion is not overselective. It is shown also that the criterion is not underselective. The case, where discounted per-period costs converge to zero but discounted total costs diverge to infinity, is considered in detail, and the sufficient conditions are proven: under not restrictive assumptions overtaking optima exist since every efficient solution is overtaking.