Reduction of Markov chains with two-time-scale state transitions

DOI10.1080/17442508.2015.1036433MaRDI QIDQ2803413zbMATH OpenOpenAlexFDO

Authors Chen Jia

Publication date 4 May 2016

Published in Stochastics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1311.2196

zbMATH Keywords

approximationlimit behaviourMarkov processessingular perturbationsstate transitionstime scales

Mathematics Subject Classification ID

Central limit and other weak theorems (60F05) Continuous-time Markov processes on discrete state spaces (60J27) Applications of continuous-time Markov processes on discrete state spaces (60J28) Time-scale analysis and singular perturbations in control/observation systems (93C70)

Abstract: In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as $lambdaightarrowinfty$. We divide the state space of the Markov chain $X$ into a fast state space and a slow state space and define a reduced chain $Y$ on the slow state space. Our main result is that the distribution of the original chain $X$ will converge in total variation distance to that of the reduced chain $Y$ uniformly in time $t$ as $lambdaightarrowinfty$.

Recommendations

• Two-Time-Scale Markov Chains and Applications to Quasi-Birth-Death Queues
• Discrete-time Markov chains with two-time scales and a countable state space: limit results and queueing applications
• scientific article; zbMATH DE number 843059
• Metastability of finite state Markov chains: a recursive procedure to identify slow variables for model reduction
• On a new method of Markov chain reduction

Cites work

• Analysis of Stiff Markov Chains
• Asymptotic Expansions of Singularly Perturbed Systems Involving Rapidly Fluctuating Markov Chains
• Asymptotic properties of a singularly perturbed Markov chain with inclusion of transient states.
• Averaging in dynamical systems and large deviations
• Limit behavior of two-time-scale diffusions revisited
• Mathematical physiology
• On large deviations in the averaging principle for SDEs with a ``full dependence
• Probability. Theory and examples.
• Singularly perturbed Markov chains: Convergence and aggregation
• Singularly perturbed Markov chains: limit results and applications

Cited in

(17)

• Using state reduction for computing steady state vectors in Markov chains of \(M/G/1\) type
• On the reduction property of the number of \(H\)-equivalent tuples of states in a discrete Markov chain
• Fluctuations of the empirical measure of freezing Markov chains
• Segregating Markov chains
• Martingale structure for general thermodynamic functionals of diffusion processes under second-order averaging
• Fast reactions with non-interacting species in stochastic reaction networks
• Metastability of finite state Markov chains: a recursive procedure to identify slow variables for model reduction
• State reduction in a Markov decision process
• scientific article; zbMATH DE number 1051717 (Why is no real title available?)
• Reduction techniques for discrete-time Markov chains on totally ordered state space using stochastic comparisons
• Two-Time-Scale Markov Chains and Applications to Quasi-Birth-Death Queues
• scientific article; zbMATH DE number 1894346 (Why is no real title available?)
• Kinetic Foundation of the Zero-Inflated Negative Binomial Model for Single-Cell RNA Sequencing Data
• Realization theory for two-time-scale distributions through approximation of Markov parameters
• Sequences of random matrices modulated by a discrete-time Markov chain*
• Analytical Time-Dependent Distributions for Gene Expression Models With Complex Promoter Switching Mechanisms
• Simplification of irreversible Markov chains by removal of states with fast leaving rates

This page was built for publication: Reduction of Markov chains with two-time-scale state transitions

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2803413)