Stationary Distributions of Discrete-Time Markov Chains in Random Environment: Exact Computations and Bounds (Q4451558)

Let \(\pi=(\pi(e,x) : e\in E, x\in X)\) be the joint stationary distribution of a Markov chain on \(E\times X\) with transition probabilities given by \[ p((e,x),(e',x')) = \begin{cases} q(e)p_{X| E}(x,x'\mid e) + r(e) p_E(e,e) \alpha(x'\mid e,x,e) &\text{ if }e'=e,\\ r(e) p_E(e,e')\alpha(x'\mid e,x,e') &\text{ if }e'\neq e,\end{cases} \] \(e\) is the current environmental state and \(r(e) = 1- q(e)\) is the probability that there is an environmental change. The first result is an inversion formula which gives \(\pi(e,x)\) in terms of a \(q(e)\)-potential and \(\pi_{a(e)}(x)\), the probability that chain is at \(x\) just after an environmental change to \(e\). This serves as the basis for an efficient computational procedure for obtaining explicit expressions of the stationary distributions of many models. Necessary and sufficient conditions are given for the stationary distribution to be of the form \(\pi(e,x)=\pi_E(e)\pi_{X\mid E}(x\mid e)\). Product-form stationary distribution can be used to approximate the stationary distribution of intricate Markov chains in slowly varying random environment. The paper ends with expressions for error bounds and three illustrations of the applicability of the method introduced. The first one is the early arrival queue with periodically varying arrival/service probabilities.