Markov Chains (Q4337696)

This is a monograph on the theory and applications of stochastic processes with an emphasis on discrete- and continuous-time Markov chains. The book is divided into five chapters. The first one is on discrete-time Markov chains. Apart from standard material it contains a section on time reversal and random walks on graphs. The following couple of chapters deal with continuous-time Markov chains. The first of the two provides a thorough introduction including the construction of minimal chains and the forward and backward equations. The second of the two chapters is largely independent of the first one and is set up in analogy with the chapter on discrete chains, i.e. it contains sections on the classification of states, invariant measures, convergence to equilibrium and ergodic theorems. The fourth chapter contains an introduction to discrete-time martingales, potential theory for Markov chains and an application to electrical networks as well as an introduction to Brownian motion including a proof of Wiener's theorem. The final chapter contains applications of the theory, e.g. to biological models, queueing models and resource management as well as an introduction to Markov decision processes. The book is designed for students with some background in (elementary) probability but not necessarily in measure theory (some basic facts are collected in an appendix). In spite of the large number of monographs on stochastic processes this one certainly fills a gap: it is a rigorous yet lively and easy-to-follow treatment of the subject with lots of diagrams, examples and heuristic explanations.