No label defined (Q5599448)

The treatise is divided in two chapters. The first chapter treats models with countable state space and arbitrary action space. The problem of non-stationary dynamic programming under risk is formulated as an appropriate tupel \((A, S, (D_n), (p_n), (r_n))\), where \(S\) is a countable state space, \(A\) is an action space, \(D_n(h)\) is the set of actions available to us if at time \(n\) the history \(h=(s_1, a_1, \ldots, s_n) \in (S \times A \times S \cdots \times S)\) has occurred, \(p_0 (s_1)\) is the initial distribution, \(p_n(h, a, s)\) for \(n \ge 1\) is the probability to be in state \(\underline{s}\) at time \((n+1)\), if at time \(\underline{n}\) the history \(h\) has occurred and if we take action \(\underline{a}\), finally \(r_n(h, a)\) is the reward given to us at time \((n+1)\). Time is discrete in the whole book. Further it is assumed that either all reward functions are ''essentially positive'' or ''essentially negative''; the notion of \(\bar{p}\)-optimality is introduced. The relation between Bellman's principle of optimality and the optimality equation derived in the book, is studied. Further a number of optimality criteria are described and the important concept of sufficient statistics is defined. The author then confines the treatment to Markovian stationary models and derives some results on so-called models with incomplete information and randomized plans. At the end of Chapter I the analysis of dynamic programming under uncertainty by means of the Bayesian approach is brought. In Chapter II the author investigates models, where the state space and action space are Borel subsets of complete separable metric spaces. The results analogous to those derived in Chapter I are obtained for such type of models. The arguments already used in Chapter I are not repeated and the author concentrates on the measure-theoretic aspects of the problem. The whole Chapter II assumes the reader has a good background in measure theory and topology. To those who have only elementary knowledge in these fields three Appendices are devoted.