Stopping Markov processes and first path on graphs (Q2496876)

Motivated by problems in finance and medicine, among others, the authors consider a strongly stationary (discrete or continuous) Markov chain \(X_n\) on a finite set \(E\) with transition matrix \(P\). Given a set of finite stopping rules satisfying some conditions, the problem of finding the desired stopping law may be solved by embedding \(X_n\) into a Markov chain on a larger state set, the tree made by both the states and the stopping rules. The desired law is then obtained from the transition matrix of the new Markov chain. Unfortunately this new Markov chain may be so big that numerical computations may not be practicable. The problem then is to find a way of compressing the oversized information by embedding a graph into a larger but minimal graph. This is what the authors presently do.