A direct projection method for Markov chains (Q1434408)

Let \(P\) be an \(n\times n\) transition probability matrix of a finite ergodic Markov chain. The author explains how the direct method of \textit{E. W. Purcell} [J. Math. Phys. 32, 180--183 (1953; Zbl 0051.35303)] based on oblique projections can be used to find the stationary distribution vector, i.e., the unique row vector \(\pi=(\pi_ 1,\ldots,\pi_ n)\) satisfying \(\pi=\pi P\), \(\pi>0\), \(\sum_ i\pi_ i=1\). The method can be also used for computation of the group inverse of the generator matrix and for updating the latter and stationary vector after a one-row change of the transition matrix. The relationship between this method and Gaussian elimination method is exploited to develop a more stable, GTH-like variant [cf. \textit{W. K. Grassmann, M. I. Taksar}, and \textit{D. P. Heyman}, Oper. Res. 33, 1107--1116 (1985; Zbl 0576.60083)] that can handle nearly uncoupled systems and other situations where the original algorithm fails.