Vector-valued semi-Markovian decision programming
This paper deals with vector-valued semi-Markov decision processes in which: \(S\) is a non-empty countable state set; for each \(i\in S\), \(A(i)\) is a finite action set; \(\alpha\) is a discount parameter; \(\beta_ \alpha(i,a)\) is the expected discounted time between state transitions, given \(\alpha\), \(i\in S\), \(a\in A(i)\); \(r(i,a)\) is the vector-valued reward between successive transitions, given \(i\in S\), \(a\in A(i)\); \(q(j\mid i,a)\) is the transition probability for \(j\in S\), given \(i\in S\), \(a\in A(i)\). Three assumptions are made in order to ensure boundedness in the subsequent analysis. For any policy \(\pi\) in the set of all policies \(\Pi\), \(V_ \alpha(\pi,i)\) is the infinite horizon expected discounted vector-valued reward if the initial state is \(i\in S\). A policy \(\pi^*\in\Pi\) is called efficient in \(\Pi\) with respect to \(i\) if, whenever \(\pi\in\Pi\) and \(V_ \alpha(\pi,i)\geq V_ \alpha(\pi^*,i)\), then \(V_ \alpha(\pi,i)=V_ \alpha(\pi^*,i)\). \(\pi^*\) is called optimal if it is efficient for all \(i\in S\). If \(V_ \alpha(\pi^*,i)\geq V_ \alpha(\pi,i)\) for all \(i\in S\), \(\pi\in\Pi\), then \(\pi^*\) is called strongly optimal. Six theorems and two algorithms are given, where Theorems 1-3 relate to the scalar case, \(r(i,a)\in R\), \(i\in S\), \(a\in A(i)\), and Theorems 4-6 relate to the vector case. Theorem 1 gives the existence of a stationary optimal policy \(\pi^*\), satisfying, for all \(i\in S\): \[ V_ \alpha(\pi^*,i)=\max_{a\in A(i)}\left[r(i,a)+\beta_ \alpha(i,a)\sum_ jq(j\mid i,a)V_ \alpha(\pi^*,j)\right]. \] Theorems 2, 3 give some optimality results in terms of the histories of the process. Theorem 4 demonstrates that a specific algorithm 1 produces an optimal policy (the algorithm works, in effect, by finding the intersection of the efficient policy sets for each \(i\in S)\). Theorem 5 leads to the existence of a stationary optimal policy. Theorem 6 gives conditions for the existence of a strongly optimal policy, and algorithm 2 allows one to detect one if it exists.
- Semi-Markovian decision models with vector-valued reward
- A new class of policies in vector-valued Markov decision processes
- scientific article; zbMATH DE number 1306281
- Optimal stationary policies in the vector-valued Markov decision process
- Optimal policies about discounted vector-valued Markovian decision programming and its algorithms
- Optimal stationary policies in the vector-valued Markov decision process
- A note on the structure of value spaces in vector-valued Markov decision processes
- Optimal policies about discounted vector-valued Markovian decision programming and its algorithms
- Weighted discounted Markov decision processes with perturbation
- Vector-valued Markov decision processes and the systems of linear inequalities
- A new class of policies in vector-valued Markov decision processes
- Semi-Markov decision processes with vector pay-offs
- Vector-valued Markov decision processes with average reward criterion: the multichain case
- The bellman equation for vector-valued semi-markovian dyanmic programiing
- VMDP-program for solving optimality problems in vector criterion Markov and semi-Markov decision processes
- On a Continuously Discounted Vector Valued Markov Decision Process
- scientific article; zbMATH DE number 3898637 (Why is no real title available?)
- scientific article; zbMATH DE number 4003943 (Why is no real title available?)
- Arbitrary state semi-Markov decision processes
- scientific article; zbMATH DE number 1306281 (Why is no real title available?)
- scientific article; zbMATH DE number 1176128 (Why is no real title available?)
- scientific article; zbMATH DE number 1778402 (Why is no real title available?)
- Semi-Markovian decision models with vector-valued reward
This page was built for publication: Vector-valued semi-Markovian decision programming
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1179954)
