CTMDP and its relationship with DTMDP
The discounted continuous time Markov decision problem (shortly, CTMDP) discussed here is of countable state space, non-empty action sets and non-uniformly bounded transition rates. Under weak conditions we proved that the discounted CTMDP is equivalent to a discounted discrete time Markov decision problem with discount factor having the form \(\beta\) (i) in the following meanings: i) their optimality equations are equal to each other: ii) their discounted objectives are equal to each other in the stochastic stationary policy set \(\Pi_ s\). Here, sup \(\beta\) (i) may be equal to 1. But when the transition rate q(j\(| i,a)\) of CTMDP is uniformly bounded, \(\beta\) (i) can be taken as a constant. By this equivalence most results known for discounted DTMDP can be easily generalized to discounted CTMDP. In addition a previous result in proving the optimality equation of discounted CTMDP is corrected.
- scientific article; zbMATH DE number 62493
- On some continuous time discounted Markov decision process.
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
- The transformation method for continuous-time Markov decision processes
- scientific article; zbMATH DE number 4059150
- Discounted and average Markov decision processes with unbounded rewards: New conditions
- The transformation method for continuous-time Markov decision processes
- scientific article; zbMATH DE number 62493 (Why is no real title available?)
- Reduction of discounted continuous-time MDPs with unbounded jump and reward rates to discrete-time total-reward mdps
This page was built for publication: CTMDP and its relationship with DTMDP
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q914561)
