Continuous-time Markov chains and applications. A two-time-scale approach (Q424648)

This book is the expanded second edition of ``Continuous-time Markov chains and applications. A singular perturbation approach.'' which appeared 1998. The main structure has been maintained; in two occasions, two former chapters have been merged, and two new chapters have been added, hence the excellent review of the first edition remains essentially valid (see [Zbl 0896.60039]). A first part starts after an introduction and a an exhaustive overview over the text followed by a chapter on background and introductory material. It closes in Chapter 3 with many motivating applications of continuous time Markov chains (cMC). The second part kicks of with a chapter on the asymptotic expansion of the forward Kolmogorov equations of singularly perturbed cMC in different settings. Chapter 5 merges two formerly separate chapters on occupation measures and ramifications. A short new Chapter 6 on asymptotic expansions for backward (Kolmogorov) equations on finite states via dual formulations of the forward case. The third part is dedicated to Markov decision problems, the stochastic control of (finite-dimensional) dynamical systems, numerical methods for control and optimization. Chapter 10 is new and gives a convergence result for the HJB equations of the asymptotic optimal control of linear quadratic Gaussian systems. The book remains clearly of interest to researchers in stochastic control, operation research, manufacturing system, engineering, economics and applied mathematics.