Singularly perturbed Markov chains: Convergence and aggregation (Q1975524)

This work develops asymptotic results for singularly perturbed Markov chains. The attention is concentrated on the solution of forward equations, related occupation measures and aggregated processes. The increasing demands on modelling and control of dynamical systems have been made for treating singularly perturbed Markov chains (see V. V. Anisimov (1995), S. P. Sethi and Q. Zhang (1994) among others). For many applications such as communication systems and queueing networks it is difficult to obtain analytic solutions. Direct numerical approximations and straightforward uses of numerical packages may not be possible due to the dimensionality. As an alternative, one decomposes a large dimensional job in small pieces [see \textit{H. A. Simon} and \textit{A. Ando}, Econometrica 29, 111-138 (1961; Zbl 0121.15103)]. Using a singular perturbation approach one separates the states in accordance with their rates of changes. Nonhomogeneous Markov chains and singularly perturbed systems arise in the framework of simulated annealing and global optimization [see \textit{T.-S. Chiang} and \textit{Y. Chow}, Ann. Probab. 17, No. 4, 1483-1502 (1989; Zbl 0687.60070)]. The authors answer the following questions: What can be done for generators that are not very smooth? Can pointwise convergence and convergence in \(L^2\) under the norm topology be obtained under acceptable weak conditions? It should be mentioned that two-time scale approaches and singularly perturbed stochastic systems have also been considered by S. N. Ethier and T. G. Kurtz (1996), H. J. Kushner (1990) and others with the main concerns on diffusion processes, jump diffusions, or systems involving wide-band noise. For ease of presentation, the authors use the transient case as a prototype in Sections 2-4. Section 2 contains the convergence result under the weak topology of \(L^2[0,T]\). Section 3 is dedicated to the study of pointwise convergence and convergence under the \(L^2\) norm topology. Section 4 treates the occupation measures and aggregated processes. In Section 5 further remarks are presented.