Sufficiency of Markov policies for continuous-time jump Markov decision processes
From MaRDI portal
Publication:5085140
Abstract: This paper extends to Continuous-Time Jump Markov Decision Processes (CTJMDP) the classic result for Markov Decision Processes stating that, for a given initial state distribution, for every policy there is a (randomized) Markov policy, which can be defined in a natural way, such that at each time instance the marginal distributions of state-action pairs for these two policies coincide. It is shown in this paper that this equality takes place for a CTJMDP if the corresponding Markov policy defines a nonexplosive jump Markov process. If this Markov process is explosive, then at each time instance the marginal probability, that a state-action pair belongs to a measurable set of state-action pairs, is not greater for the described Markov policy than the same probability for the original policy. These results are used in this paper to prove that for expected discounted total costs and for average costs per unit time, for a given initial state distribution, for each policy for a CTJMDP the described a Markov policy has the same or better performance.
Recommendations
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
- New discount and average optimality conditions for continuous-time Markov decision processes
- Continuous-time controlled Markov chains.
- Continuous-Time Markov Decision Processes with Discounted Rewards: The Case of Polish Spaces
- New sufficient conditions for average optimality in continuous-time Markov decision processes
Cites work
- A generalization of ‘expectation equals reciprocal of intensity' to non-stationary exponential distributions
- A Note on Memoryless Rules for Controlling Sequential Control Processes
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
- Continuous-time Markov decision processes. Borel space models and general control strategies. With a foreword by Albert Nikolaevich Shiryaev
- Continuous-time Markov decision processes. Theory and applications
- Continuously Discounted Markov Decision Model with Countable State and Action Space
- Controlled Jump Markov Models
- Controlled Markov Models with Countable State Space and Continuous Time
- Discounted continuous-time constrained Markov decision processes in Polish spaces
- Discounted continuous-time Markov decision processes with unbounded rates and randomized history-dependent policies: the dynamic programming approach
- Finite State Continuous Time Markov Decision Processes with a Finite Planning Horizon
- Finite state continuous time Markov decision processes with an infinite planning horizon
- scientific article; zbMATH DE number 3718234 (Why is no real title available?)
- scientific article; zbMATH DE number 977051 (Why is no real title available?)
- scientific article; zbMATH DE number 1834045 (Why is no real title available?)
- scientific article; zbMATH DE number 3222422 (Why is no real title available?)
- scientific article; zbMATH DE number 3108056 (Why is no real title available?)
- Multivariate point processes: predictable projection, Radon-Nikodym derivatives, representation of martingales
- Negative Dynamic Programming
- On Reducing a Jump Controllable Markov Model to a Model with Discrete Time
- On solutions of Kolmogorov's equations for nonhomogeneous jump Markov processes
- Optimal Control of a Server Farm
- Optimal switching on and off the entire service capacity of a parallel queue
- Reduction of discounted continuous-time MDPs with unbounded jump and reward rates to discrete-time total-reward mdps
- Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion
- Stochastic optimal control. The discrete time case
- Structures of optimal policies in MDPs with unbounded jumps: the state of our art
- The transformation method for continuous-time Markov decision processes
Cited in
(8)- On an extremal property of Markov chains and sufficiency of Markov strategies in Markov decision processes with the Dubins-Savage criterion
- scientific article; zbMATH DE number 4084791 (Why is no real title available?)
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
- On Forward and Backward Kolmogorov Equations for Pure Jump Markov Processes and Their Generalizations
- Discounted cost exponential semi-Markov decision processes with unbounded transition rates: a service rate control problem with impatient customers
- Risk-sensitive zero-sum games for continuous-time jump processes with unbounded rates and Borel spaces
- Control-limit policies for coordinated dispatching and preventive maintenance in multi-product manufacturing systems
- Title not available (Why is no real title available?)
This page was built for publication: Sufficiency of Markov policies for continuous-time jump Markov decision processes
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5085140)
