On the uniqueness of solutions to the Poisson equations for average cost Markov chains with unbounded cost functions (Q1416787)

Markov chains (MC) with state space \(X\) and transition matrix \(P\) are treated, to which a cost function \(c: X\to{\mathbb R}\) is associated. \(X\) is supposed as countable so that \(c\) may be unbounded. For all investigations as a basic tool, Banach spaces \(B_w(X)\) of \(w\)-bounded functions \(u: X\to{\mathbb R}\) with the norm \(\| u\| _w:=\sup_{x\in X}\frac{| u(x)| }{w(x)},\;w: X\to[1,\infty)\), will act, and \(c\in B_w(X)\) is postulated. Now it is asked under what conditions the Poisson equations \[ \varphi=P\varphi,\quad\varphi+v=c+Pv\tag{*} \] will have exactly one solution in \(B_w(X)\). At beginning of reading, the reader may think to have to do with Markov decision processes as the Poisson equations also play a role in Howard's policy improvement algorithm. But \(P\) and \(c\) are fixed. For the following, some ergodicity and recurrence properties are supposed. If the MC is geometrically \(w\)-ergodic (\(w\)-{GE}) and stable with finitely many closed classes, then \(\varphi\) is unique and \(v\) is unique up to a constant in every closed class. For this result, originally found by \textit{R. Dekker} [Ph.D. thesis, Leiden, 1985], an alternative probabilistic proof is given. The property of geometric \(w\)-recurrence (\(w\)-({GR}(\(M\))), i.e. \(\exists\varepsilon>0:| | _MP| | _w\leq1-\varepsilon,\;_MP\) being a taboo transition matrix for some \(M\subset X\), is easier to check than \(w\)-{GE} and sufficient for \(w\)-{GE}. Three queueing examples (multi-server queue, independent single server queue, priority queue with switching costs) are considered in greater detail, and appropriate taboo sets \(M\) are given so that (*) will have a unique solution.