On density estimation from ergodic processes (Q1307505): Difference between revisions

Let \(\mu\) be a probability measure on the Borel subsets \({\mathcal B }\) of the half-open unit interval \([0,1)\). A measurable transformation \(T:[0,1)\to [0,1)\) is \(\mu\)-preserving if \(\mu(T^{-1}B)=\mu(B)\) for each \(B\in{\mathcal B}\), and is ergodic if for each \(B\) such that \(T^{-1}B=B\) either \(\mu(B)=0\) or \(\mu(B)=1 \). A sequence \({\mathbf X}=\{X_1,X_2,\dots\}\) of random variables defined on \(([0,1),{\mathcal B},\mu)\) is said to be stationary and ergodic if there exists an ergodic \(\mu\)-preserving transformation \(T\) and a Borel measurable function \(g:[0,1)\to\mathbb{R}\) such that \[ X_i(\omega)= g(T^{i-1} \omega), \quad i=1,2, \dots, \] for \(\mu\)-a.e. \(\omega \in[0,1)\). If the distribution \(\nu=\mu\circ g^{-1}\) of each random variable \(X_i\) is absolutely continuous with respect to Lebesgue measure \(\lambda\), then \(X_i\) is distributed according to the probability density \(f=d \nu/d\lambda\), written \(X_i\sim f\). Estimation of \(f\) from finitely many observations of the process \({\mathbf X}\) is an important and well studied problem in applied and theoretical statistics. The density estimation problem and its potential solutions can be formalized as follows: Problem. Given an ergodic process \({\mathbf X}=X_1,X_2, \dots, \in\mathbb{R}\) with \(X_i\sim f\), select integrable functions \(\widehat f_1,\widehat f_2, \dots\) such that (i) \(\widehat f_n\) depends only on \(X_1, \dots, X_n\) and (ii) \(\int|\widehat f_n- f|dx\to 0\) in probability as \(n\to\infty\). Here, it is shown that no procedure can consistently estimate the one-dimensional marginal density of every stationary ergodic process for which such a density exists. A similar result is established for the problem of estimating the support of the marginal distribution of an ergodic process.