Discrete-time singularly perturbed Markov chains (Q4407571)

This is a survey of recent developments in singularly perturbed Markov chains in discrete time. The authors illustrate the use of singular perturbation techniques in treating large-scale discrete-time hybrid stochastic dynamic systems involving discrete event interventions. To reduce the complexity of such systems, time-scale separation and singularly perturbed Markov chains are used in the modeling and analysis. Asymptotic expansions of probability vectors and the structural properties of these Markov chains are provided. There are five sections. In Section 1 motivation and precise formulation of singularly perturbed Markov chains in discrete time are given. Section 2 proceeds with the construction of asymptotic expansions of the corresponding probability vectors and transition matrices. In Section 3 sequences of unscaled and scaled occupation measures are defined and limit properties of such sequences are derived. Proofs are only given for the case where the transition matrices satisfy so-called weak irreducibility conditions while results for transition matrices with more complex structures are presented without proofs. Also, the authors concentrate on the time-homogeneous case with a finite state space. In Section 4 the results in Sections 2 and 3 are extended to more complex situations in which multi-block weakly irreducible matrices are considered. Possible future research topics are mentioned in Section 5. There are 65 references out of which 29 belong to the authors, alone or in collaboration. For the entire collection see [Zbl 1005.00025].