A sandwich proof of the Shannon-McMillan-Breiman theorem (Q1107656): Difference between revisions

Let \(X_ 0,X_ 1,..\). be a stationary ergodic process with Polish state space S. Let M be a reference measure on the space of all one-sided infinite sequences from S. Suppose that M is finite order Markov with stationary transition kernel. Further, suppose that for each positive integer n, the distribution \(P_ n\) of \((X_ 0,X_ 1,...,X_{n-1})\) is absolutely continuous with respect to the marginal distribution \(M_ n\) that one obtains from M by projecting on the first n coordinates. Let \(\{f_ n\}\) be the sequence of random variables \(f_ n=(dP_ n/dM_ n)(X_ 0,x_ 1,...,X_{n-1}),\quad n=1,2,...\quad.\) It is known that the sequence \(\{n^{-1} \log f_ n\}\) converges almost surely. (This result is a generalization of the Shannon-McMillan-Breiman theorem.) The authors provide a new proof of this result which involves an ingenious ``sandwiching'' of the limit superior and limit inferior of \(\{n^{-1} \log f_ n\}\) between two random variables that have the same expected value.