scientific article; zbMATH DE number 3854831

From MaRDI portal
Publication:3323724





[For the entire collection see Zbl 0531.00015.] - The following Markov decision process is considered: the state set S is countable; decisions are made at discrete points in time \(t=0,1,2...,\); each feasible action set A(i), \(i\in S\), is compact and metric; the reward \(c_ i(\cdot)\) and transition probabilities \(p_{ij}(\cdot)\) are continuous on A(i) for \(i\in S\), and \(| c_ i(a)| \leq M\), \(\forall i\in S\), \(a\in A(i)\), for some \(M<\infty\). It is required to find a policy \(\pi =(f_ 0,f_ 1,...f_ t...)\) where the \(\{f_ t\}\) are decision rules on S, with \(f_ t\in F\), the set of all decision rules, to \[ _{\pi}[\lim \sup_{n\to \infty}1/(n+1)\sum^{n}_{k=0}(\prod^{k-1}_{t=0}P(f_ t)c(f_ k)=g(\pi)] \] where P(f) is the transition matrix for decision rule f, and, for \(k=0\) the product expression is the identity matrix. Five equivalent conditions are considered, the first one being for each f the matrix \[ \Pi(f)=\lim_{n\to \infty}1/(n+1)\sum^{n}_{k=0}P^ k(f) \] is stochastic, P(f) has a finite set of irreducible subsets of S, and \[ \lim_{n\to \infty}\sup_{f}\| P^ n(f)-\Pi(f)\| -0 \] where \(\| \cdot \|\) is the supremum norm. Under this condition (or one of its four equivalents) a minimising stationary policy \(\pi =(f,f,f,...)\) exists where f satisfies the optimality equations \[ \min_{f\in F}[P(f)g]=g\quad and\quad \min_{f\in F_ 1}[c(f)+P(f)w]=w+g \] for some \(w\in {\mathbb{R}}^{\infty}\), \(g\in {\mathbb{R}}^{\infty}\), and, for such solutions \(g=\min_{\pi}[g(\pi)]\). The finiteness of the number of irreducible sets may be relaxed in the stipulated condition. No proofs are given. The paper is a precis of The optimality equation in multichain denumerable state Markov decision processes with the average cost criterion: The bounded cost case, A. E. Report 2/82, Univ. of Amsterdam (1982).











This page was built for publication:

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3323724)