A finite characterization of weak lumpable Markov processes. I: The discrete time case (Q1177209)

The paper considers a Markov chain \((X_ k)\) on a finite state space. The goal of the paper may be formulated as follows: For a fixed function \(\psi\) on the state space, under what conditions is the stochastic process \((\psi(X_ k))\) Markov? Interestingly, this depends not only on the function \(\psi\), but on the initial distribution of the Markov chain \((X_ k)\). The purpose of the paper is to characterize the set of all initial distributions under which \((\psi(X_ k))\) is Markov, which is denoted \({\mathcal A_ M}\), and to construct a finite algorithm which computes these initial distributions. Much of the paper reviews the authors' prior work. The core of the paper is Section 3 where the algorithm is constructed. This algorithm is based upon a refinement of their earlier characterization of \({\mathcal A_ M}\). By setting up the problem in the framework of convex analysis the authors are able to obtain a short proof that this set is equal to a finite intersection of elementary sets of initial distributions, and from this the algorithm is easily obtained. The paper is concluded with an example to illustrate the methods.