Continuous-time controlled Markov chains.
From MaRDI portal
The authors consider continuous-time Markov decision processes with a denumerable state space and Borel action space where the cost and the transition rates are allowed to be unbounded. In the class of deterministic stationary policies they first give very weak conditions under which the existence of \(\varepsilon\)-optimal \((\varepsilon\geq 0)\) policies is proved using the construction of a minimum \(Q\)-process. In the class of randomized Markov policies they secondly present a new necessary and sufficient condition for a nonhomogeneous \(Q\)-process to be regular and under this condition the existence of \(\varepsilon\)-optimal stationary policies is proved. In the case of nonregular \(Q\)-process, they further use Kolmogorov's forward equation for a nonhomogeneous minimum \(Q\)-process to prove the existence of \(\varepsilon\)-optimal stationary policies.
Recommendations
- Continuous-time controlled Markov chains with discounted rewards
- scientific article; zbMATH DE number 1829768
- scientific article; zbMATH DE number 4081328
- Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates
- Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
Cites work
- A note on bias optimality in controlled queueing systems
- A note on optimality conditions for continuous-time Markov decision processes with average cost criterion
- A probabilistic analysis of bias optimality in unichain Markov decision processes
- Bias Optimality in Controlled Queueing Systems
- Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach
- Continuous-time Markov chains. An applications-oriented approach
- Continuously Discounted Markov Decision Model with Countable State and Action Space
- Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
- Finite state continuous time Markov decision processes with an infinite planning horizon
- scientific article; zbMATH DE number 3151099 (Why is no real title available?)
- scientific article; zbMATH DE number 1233798 (Why is no real title available?)
- scientific article; zbMATH DE number 700091 (Why is no real title available?)
- scientific article; zbMATH DE number 941166 (Why is no real title available?)
- scientific article; zbMATH DE number 805121 (Why is no real title available?)
- scientific article; zbMATH DE number 921048 (Why is no real title available?)
- scientific article; zbMATH DE number 2189771 (Why is no real title available?)
- On Homogeneous Markov Models with Continuous Time and Finite or Countable State Space
- On the Integro-Differential Equations of Purely Discontinuous Markoff Processes
- On three classical problems for Markov chains with continuous time parameters
- Optimal Control of a Birth and Death Epidemic Process
- Optimal control of random walks, birth and death processes, and queues
- Optimal stationary policies for denumerable Markov chains in continuous time
- Value iteration in a class of average controlled Markov chains with unbounded costs: necessary and sufficient conditions for pointwise convergence
Cited in
(34)- The stopped distributions of a controlled Markov chain with discrete time
- Continuous-time controlled Markov chains with discounted rewards
- Optimal control of Markov processes with age-dependent transition rates
- Imprecise continuous-time Markov chains
- First passage models for denumerable semi-Markov decision processes with nonnegative discounted costs
- Blackwell optimality in the class of Markov policies for continuous-time controlled Markov chains
- Control of continuous-time Markov chains with safety constraints
- New discount and average optimality conditions for continuous-time Markov decision processes
- Selected topics on continuous-time controlled Markov chains and Markov games
- scientific article; zbMATH DE number 4204268 (Why is no real title available?)
- Bias Optimality for Continuous-Time Controlled Markov Chains
- Randomized and relaxed strategies in continuous-time Markov decision processes
- scientific article; zbMATH DE number 3950347 (Why is no real title available?)
- scientific article; zbMATH DE number 3956845 (Why is no real title available?)
- scientific article; zbMATH DE number 4081328 (Why is no real title available?)
- Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates
- scientific article; zbMATH DE number 2062355 (Why is no real title available?)
- scientific article; zbMATH DE number 6913921 (Why is no real title available?)
- Continuous-time controlled jump Markov processes on the finite horizon
- Realizable strategies in continuous-time Markov decision processes
- Kolmogorov's equations for jump Markov processes and their applications to control problems
- The existence of optimal control for continuous-time Markov decision processes in random environments
- Sufficiency of Markov policies for continuous-time jump Markov decision processes
- The limit behaviour of imprecise continuous-time Markov chains
- Zero-sum games for continuous-time jump Markov processes in Polish spaces: discounted payoffs
- Continuous-Time Markov Decision Processes with Unbounded Transition and Discounted-Reward Rates
- Dynamic control of a single-server system with abandonments
- Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
- Discounted continuous-time constrained Markov decision processes in Polish spaces
- Continuous-time Markov decision processes with state-dependent discount factors
- Zero-sum continuous-time Markov games with unbounded transition and discounted payoff rates
- Existence and regularity of a nonhomogeneous transition matrix under measurability conditions
- A survey of recent results on continuous-time Markov decision processes (with comments and rejoinder)
- Average optimality for continuous-time Markov decision processes in Polish spaces
This page was built for publication: Continuous-time controlled Markov chains.
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1872339)
