Trajectory averaging for stochastic approximation MCMC algorithms (Q605929)

The asymptotic efficiency of the trajectory averaging estimator for the stochastic approximation Markov Chain Monte Carlo algorithm (SAMCMC) is proved. In general, the stochastic approximation algorithm is an iterative stochastic method to solve equations of the type \[ h(\theta)=\int H(x,\theta)f_\theta(x)\,dx=0 \] where \(f\) is a density depending on the parameter \(\theta\). Estimates \(\theta_k\) of the solution \(\theta^\ast\) are obtained by the following iteration procedure: Simulate \(X_{k+1}\sim f_{\theta_k}(x)\) and set \(\theta_{k+1}=\theta_k+a_k H(\theta_k,X_{k+1})\) where \(a_k\) are choosen gain factors. General SAMCMC algorithms are versions of this iteration procedure where the sample \(X_{k+1}\) is drawn with respect to a Markov transition kernel \(P_{\theta_k}\) which admits \(f_{\theta_k}\) as an invariant distribution. The trajectory averaging estimator is given by \(\overline{\theta}_n=\sum_{k=1}^n \theta_k/n\). The authors propose a series of conditions on the functions \(h\) and \(H\), the Markov kernels \(P_\theta\) and the gain factors \(a_k\) which allow them to infer the main results: the SAMCMC algorithm converges almost surely, i.e., \(\theta_k\to\theta^\ast\) a.s., and the trajectory averaging estimator is asymptotically efficient, i.e., \(\sqrt{n}(\overline{\theta}_n-\theta^\ast)\to N(0,\Gamma)\) in distribution where \(\Gamma\) is the minimal variance. This result is then applied to the stochastic approximation Monte Carlo algorithm and the stochastic approximation MLE algorithm.