scientific article; zbMATH DE number 3854831
[For the entire collection see Zbl 0531.00015.] - The following Markov decision process is considered: the state set S is countable; decisions are made at discrete points in time \(t=0,1,2...,\); each feasible action set A(i), \(i\in S\), is compact and metric; the reward \(c_ i(\cdot)\) and transition probabilities \(p_{ij}(\cdot)\) are continuous on A(i) for \(i\in S\), and \(| c_ i(a)| \leq M\), \(\forall i\in S\), \(a\in A(i)\), for some \(M<\infty\). It is required to find a policy \(\pi =(f_ 0,f_ 1,...f_ t...)\) where the \(\{f_ t\}\) are decision rules on S, with \(f_ t\in F\), the set of all decision rules, to \[ _{\pi}[\lim \sup_{n\to \infty}1/(n+1)\sum^{n}_{k=0}(\prod^{k-1}_{t=0}P(f_ t)c(f_ k)=g(\pi)] \] where P(f) is the transition matrix for decision rule f, and, for \(k=0\) the product expression is the identity matrix. Five equivalent conditions are considered, the first one being for each f the matrix \[ \Pi(f)=\lim_{n\to \infty}1/(n+1)\sum^{n}_{k=0}P^ k(f) \] is stochastic, P(f) has a finite set of irreducible subsets of S, and \[ \lim_{n\to \infty}\sup_{f}\| P^ n(f)-\Pi(f)\| -0 \] where \(\| \cdot \|\) is the supremum norm. Under this condition (or one of its four equivalents) a minimising stationary policy \(\pi =(f,f,f,...)\) exists where f satisfies the optimality equations \[ \min_{f\in F}[P(f)g]=g\quad and\quad \min_{f\in F_ 1}[c(f)+P(f)w]=w+g \] for some \(w\in {\mathbb{R}}^{\infty}\), \(g\in {\mathbb{R}}^{\infty}\), and, for such solutions \(g=\min_{\pi}[g(\pi)]\). The finiteness of the number of irreducible sets may be relaxed in the stipulated condition. No proofs are given. The paper is a precis of The optimality equation in multichain denumerable state Markov decision processes with the average cost criterion: The bounded cost case, A. E. Report 2/82, Univ. of Amsterdam (1982).
- scientific article; zbMATH DE number 3900546
- Solution to the optimality equation in a class of Markov decision chains with the average cost criterion
- A new condition for the existence of optimal stationary policies in average cost Markov decision processes
- Functional characterization for average cost Markov decision processes with Doeblin's conditions
- Average, Sensitive and Blackwell Optimal Policies in Denumerable Markov Decision Chains with Unbounded Rewards
- Optimal switching problem for countable Markov chains: Average reward criterion
- Semi-infinite Markov decision processes
- Computing semi-stationary optimal policies for multichain semi-Markov decision processes
- scientific article; zbMATH DE number 3876943 (Why is no real title available?)
- Multichain Markov Decision Processes with a Sample Path Constraint: A Decomposition Approach
- Solution to the optimality equation in a class of Markov decision chains with the average cost criterion
- A Rollout Algorithm for Multichain Markov Decision Processes with Average Cost
- scientific article; zbMATH DE number 3900546 (Why is no real title available?)
- A value iteration method for undiscounted multichain Markov decision processes
- The Existence of a Minimum Pair of State and Policy for Markov Decision Processes under the Hypothesis of Doeblin
- scientific article; zbMATH DE number 1908240 (Why is no real title available?)
- Denumerable continuous-time Markov decision processes with multiconstraints on average costs
- An Algorithm to Identify and Compute Average Optimal Policies in Multichain Markov Decision Processes
- On Average Reward Semi-Markov Decision Processes with a General Multichain Structure
This page was built for publication:
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3323724)