Singularly perturbed Markov chains: Convergence and aggregation

DOI10.1006/JMVA.1999.1855MaRDI QIDQ1975524zbMATH OpenOpenAlexFDO

Authors Q. Zhang, G. Badowski, George Yin

Publication date 18 December 2000

Published in Journal of Multivariate Analysis (Search for Journal in Brave)

Full work available at URL https://doi.org/10.1006/jmva.1999.1855

convergence Markov chains simulated annealing

Asymptotic expansions of solutions to ordinary differential equations (34E05) Functional limit theorems; invariance principles (60F17)

Review text

This work develops asymptotic results for singularly perturbed Markov chains. The attention is concentrated on the solution of forward equations, related occupation measures and aggregated processes. The increasing demands on modelling and control of dynamical systems have been made for treating singularly perturbed Markov chains (see V. V. Anisimov (1995), S. P. Sethi and Q. Zhang (1994) among others). For many applications such as communication systems and queueing networks it is difficult to obtain analytic solutions. Direct numerical approximations and straightforward uses of numerical packages may not be possible due to the dimensionality. As an alternative, one decomposes a large dimensional job in small pieces [see \textit{H. A. Simon} and \textit{A. Ando}, Econometrica 29, 111-138 (1961; Zbl 0121.15103)]. Using a singular perturbation approach one separates the states in accordance with their rates of changes. Nonhomogeneous Markov chains and singularly perturbed systems arise in the framework of simulated annealing and global optimization [see \textit{T.-S. Chiang} and \textit{Y. Chow}, Ann. Probab. 17, No. 4, 1483-1502 (1989; Zbl 0687.60070)]. The authors answer the following questions: What can be done for generators that are not very smooth? Can pointwise convergence and convergence in \(L^2\) under the norm topology be obtained under acceptable weak conditions? It should be mentioned that two-time scale approaches and singularly perturbed stochastic systems have also been considered by S. N. Ethier and T. G. Kurtz (1996), H. J. Kushner (1990) and others with the main concerns on diffusion processes, jump diffusions, or systems involving wide-band noise. For ease of presentation, the authors use the transient case as a prototype in Sections 2-4. Section 2 contains the convergence result under the weak topology of \(L^2[0,T]\). Section 3 is dedicated to the study of pointwise convergence and convergence under the \(L^2\) norm topology. Section 4 treates the occupation measures and aggregated processes. In Section 5 further remarks are presented.

Recommendations

Cites work

Cited in

(18)

This page was built for publication: Singularly perturbed Markov chains: Convergence and aggregation

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1975524)