scientific article; zbMATH DE number 3908159

From MaRDI portal
Publication:3686493





The objective of sequential Markovian decision (MD) making for a process with a finite number of states is to determine the optimal strategies on a finite or infinite planning horizon. To each state i, \(i\in \overline{1,N}\), is associated a finite number \(k_ i\) of alternatives. The probabilities \(p^ k_{ij}\), \(1\leq k\leq k_ i\), i,j\(\in \overline{1,N}\), of a transition from the state i into the state j, by one step, using the alternative \(k\in \overline{l,k_ i}\) are known. The decision associated to the state i at the moment n is defined as an alternative \(d_ i(n)\in \overline{l,k_ i}\), and the policy at the moment n is the vector \(d(n)=(d_ i(n))\), \(i\in \overline{l,N}\). The general problem is to find the optimal strategies \(s^*(n)=(d^*(0),d^*(l),...,d^*(n-l))\) or \(s^*=(d^*(0),d^*(l),...,d^*(n-l),...)\) which maximize some total expected reward or which minimize some total expected loss. This work represents an outgrowth of the author's thesis and is also a survey of the results obtained in this domain by other mathematicians; the first monograph on this domain is due to \textit{R. A. Howard} [Dynamic programming and Markov processes (1960; Zbl 0091.16001)]. The book has seven chapters: I. General aspects of MD problems; II. Examples of MD problems; III. General results in the study of problems; IV. MD problems with a discount factor; V. The linear problem associated to a MD problem; VI. Continuous-time MD problems; VII. Some MD problems solved by algorithms presented in the book. All notions and the notation are well defined; all proofs are given.











This page was built for publication:

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3686493)