Existence and regularity of a nonhomogeneous transition matrix under measurability conditions

DOI10.1007/S10959-008-0163-9zbMATH Open1147.60050arXiv0804.4441OpenAlexW2013329239MaRDI QIDQ939136FDOQ939136

Authors: Liuer Ye, Xianping Guo, Onésimo Hernández-Lerma 

Publication date: 21 August 2008

Published in: Journal of Theoretical Probability (Search for Journal in Brave)

Abstract: This paper is about the existence and regularity of the transition probability matrix of a nonhomogeneous continuous-time Markov process with a countable state space. A standard approach to prove the existence of such a transition matrix is to begin with a continuous (in t) and conservative matrix Q(t)=[q_{ij}(t)] of nonhomogeneous transition rates q_{ij}(t), and use it to construct the transition probability matrix. Here we obtain the same result except that the q_{ij}(t) are only required to satisfy a mild measurability condition, and Q(t) may not be conservative. Moreover, the resulting transition matrix is shown to be the minimum transition matrix and, in addition, a necessary and sufficient condition for it to be regular is obtained. These results are crucial in some applications of nonhomogeneous continuous-time Markov processes, such as stochastic optimal control problems and stochastic games, which motivated this work in the first place.

Full work available at URL: https://arxiv.org/abs/0804.4441

Recommendations

• Construction and regularity of transition functions on Polish spaces under measurability conditions
• scientific article; zbMATH DE number 7113325
• On \(L_1\)-weak ergodicity of nonhomogeneous continuous-time Markov processes
• On solutions of Kolmogorov's equations for nonhomogeneous jump Markov processes
• Quasi-ergodicity for non-homogeneous continuous-time Markov chains

zbMATH Keywords

Kolmogorov equationsminimum transition matrixnonhomogeneous continuous-time Markov chainsnonhomogeneous transition rates

Mathematics Subject Classification ID

Continuous-time Markov processes on discrete state spaces (60J27) Transition functions, generators and resolvents (60J35)

Cites Work

• Zero-sum games for continuous-time Markov chains with unbounded transition and average payoff rates
• Title not available (Why is that?)
• Nonzero-sum games for continuous-time Markov chains with unbounded discounted payoffs
• Title not available (Why is that?)
• A survey of Markov decision models for control of networks of queues
• Denumerable-state continuous-time Markov decision processes with unbounded transition and reward rates under the discounted criterion
• Title not available (Why is that?)
• Drift and monotonicity conditions for continuous-time controlled markov chains with an average criterion
• Title not available (Why is that?)
• Continuous-time controlled Markov chains.
• On the Integro-Differential Equations of Purely Discontinuous Markoff Processes
• Title not available (Why is that?)
• Continuous time Markovian decision processes average return criterion
• Semi-infinite semi-Markov stochastic games.
• Nonstationary continuous time markov decision processes with the expected total rewards criterion

Cited In (15)

• Kolmogorov's equations for jump Markov processes and their applications to control problems
• On Forward and Backward Kolmogorov Equations for Pure Jump Markov Processes and Their Generalizations
• Continuous-time Markov decision processes with state-dependent discount factors
• Variance minimization and the overtaking optimality approach to continuous-time controlled Markov chains
• Asymptotic fluid optimality and efficiency of the tracking policy for bandwidth-sharing networks
• New sufficient conditions for average optimality in continuous-time Markov decision processes
• Construction and regularity of transition functions on Polish spaces under measurability conditions
• Uniform ergodicity of continuous-time controlled Markov chains: a survey and new results
• On solutions of Kolmogorov's equations for nonhomogeneous jump Markov processes
• Total reward criteria for unconstrained/constrained continuous-time Markov decision processes
• On the Existence of a Homogeneous Transition Function
• New discount and average optimality conditions for continuous-time Markov decision processes
• Denumerable continuous-time Markov decision processes with multiconstraints on average costs
• The vanishing discount approach to constrained continuous-time controlled Markov chains
• Kolmogorov's equations for jump Markov processes with unbounded jump rates

This page was built for publication: Existence and regularity of a nonhomogeneous transition matrix under measurability conditions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q939136)