Stationary distributions of perturbed Markov chains (Q1085891)

Let \(P^{(1)}\) be an irreducible Markov matrix on the finite set S and let \(\pi^{(1)}\) be the invariant probability distribution for \(P^{(1)}\), i.e., \(\pi^{(1)}\) is a nonnegative row vector satisfying \(\pi^{(1)}\cdot 1=1\), where 1 denotes a column vector of ones, and \(\pi^{(1)}P^{(1)}=\pi^{(1)}\). Let \(P^{(2)}\) be obtained by perturbing \(P^{(1)}\) via a rank 1 linear transformation: \(P^{(2)}=P^{(1)}+ab^ T\), where a and b are column vectors such that \(b^ T\cdot 1=1\), and let \(\pi^{(2)}\) be the associated invariant distribution. The author expresses \(\pi^{(2)}\) in terms of \(\pi^{(1)}\), a and b. The main result is the following: let \(\rho\),t be row and column vectors, respectively, such that \(\rho\cdot 1\neq 0\) and \(\pi^{(1)}\cdot t\neq 0\), and define \(\alpha =\rho (I-P^{(1)}+t\rho)^{-1}\) and \(\beta =b^ T(I- P^{(1)}+t\rho)^{-1}\). Then \(\pi^{(1)}=\alpha /(\alpha \cdot 1)\) and \[ \pi^{(2)}=\frac{(\alpha \cdot a)\beta +(1-\beta \cdot a)\alpha}{(\alpha \cdot a)(\beta \cdot 1)+(1-\beta \cdot a)(\alpha \cdot 1)}. \] Application is made to the special cases that \(P^{(1)}\) and \(P^{(2)}\) differ in one row only or differ in exactly two elements in the same row.