Convergence rate of Gibbs sampler and its application (Q880862)

In the article the convergence rate of different Markov chain Monte Carlo schemes is considered. Let \(X=(x_1,\dots,x_s)\) denote the random vector having density distribution \(f(x)\) and suppose that it is easy to draw samples from \(f(x_i| x_1,\dots,x_{i-1},x_{i+1},\dots,x_s)\), \(i=1,\dots,s\). To draw random vectors \(X^{(t)}\), \(t=1,2,\dots,\) from \(f(x)\) one can use the following scheme, called systematic scan Gibbs sample. Given a vector \(X^{(t)}\), in order to draw \(X^{(t+1)}\), one can draw \(x_1^{(t+1)}\) from \(f(x_1| x_2^{(t)},\dots,x_s^{(t)})\), then \(x_i^{(t+1)}\) from \(f(x_i| x_1^{(t+1)},\dots,x_{i-1}^{(t+1)},x_{i+1}^{(t)},\dots,x_s^{(t)})\), \(i=2,\dots,s-1\), and \(x_s^{(t+1)}\) from \(f(x_s| x_1^{(t+1)},\dots,x_{s-1}^{(t+1)})\). The density functions of \(X^{(t)}\), \(X^{(t+1)}\) are denoted by \(f_t(x)\), \(f_{t+1}(x)\), the transition function from \(X^{(t)}=y\) to \(X^{(t+1)}=x\) is denoted by \(K(x| y)\), so that \(f_{t+1}(x)=\int K(x| y)f_{t}(y) \,dy\). If the systematic scan Gibbs sampler is used to draw a sample from the marginal density \(f(x_1,\dots,x_{s-1})\) then the obtained sample is called a collapsed Gibbs sample. The convergence rate is defined by the Pearson \(\chi^2\)-distance between density functions \(f(x)\) and \(f_t(x)\) as \(\text{Var}_f(f_t(X)f^{-1}(X))\). Under a set of regularity conditions it is proved that the convergence rate of the systematic Gibbs sampler \(f_t(x)\) to the target distribution \(f(x)\) is the norm of a forward operator \(F\) which is defined as \(Ft(x)=E(t(X_2)| X_1=x)=\int t(y)K(y| x) \,dy\). It is discussed that the collapsed Gibbs sampler has a faster convergence rate than the systematic one as proposed by Liu et al. This result is proved quantitatively. It is also proved that the convergence rate defined with the spectral radius of the matrix by Robert and Shau is equivalent to the corresponding radius of the forward operator. Finally, an illustrative example of the systematic scan Gibbs sample is considered when the target distribution is a normal distribution.