Existence and regularity of a nonhomogeneous transition matrix under measurability conditions
From MaRDI portal
Publication:939136
Abstract: This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t) and conservative matrix Q(t)=[q_{ij}(t)] of nonhomogeneous transition rates q_{ij}(t), and use it to construct the transition probability matrix. Here we obtain the same result except that the q_{ij}(t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, which motivated this work in the first place.
Recommendations
- Construction and regularity of transition functions on Polish spaces under measurability conditions
- scientific article; zbMATH DE number 7113325
- On \(L_1\)-weak ergodicity of nonhomogeneous continuous-time Markov processes
- On solutions of Kolmogorov's equations for nonhomogeneous jump Markov processes
- Quasi-ergodicity for non-homogeneous continuous-time Markov chains
Cites work
- scientific article; zbMATH DE number 1233798 (Why is no real title available?)
- scientific article; zbMATH DE number 1159486 (Why is no real title available?)
- scientific article; zbMATH DE number 1904082 (Why is no real title available?)
- scientific article; zbMATH DE number 1416652 (Why is no real title available?)
- scientific article; zbMATH DE number 3215022 (Why is no real title available?)
- A survey of Markov decision models for control of networks of queues
- Continuous time Markovian decision processes average return criterion
- Continuous-time controlled Markov chains.
- Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
- Drift and monotonicity conditions for continuous-time controlled markov chains with an average criterion
- Nonstationary continuous time markov decision processes with the expected total rewards criterion
- Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs
- On the Integro-Differential Equations of Purely Discontinuous Markoff Processes
- Semi-infinite semi-Markov stochastic games.
- Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates
Cited in
(15)- Kolmogorov's equations for jump Markov processes and their applications to control problems
- Continuous-time Markov decision processes with state-dependent discount factors
- On Forward and Backward Kolmogorov Equations for Pure Jump Markov Processes and Their Generalizations
- Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains
- Asymptotic fluid optimality and efficiency of the tracking policy for bandwidth-sharing networks
- Construction and regularity of transition functions on Polish spaces under measurability conditions
- New sufficient conditions for average optimality in continuous-time Markov decision processes
- Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results
- On solutions of Kolmogorov's equations for nonhomogeneous jump Markov processes
- Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
- On the Existence of a Homogeneous Transition Function
- New discount and average optimality conditions for continuous-time Markov decision processes
- Denumerable continuous-time Markov decision processes with multiconstraints on average costs
- The vanishing discount approach to constrained continuous-time controlled Markov chains
- Kolmogorov's equations for jump Markov processes with unbounded jump rates
This page was built for publication: Existence and regularity of a nonhomogeneous transition matrix under measurability conditions
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q939136)
