Non-reversible Metropolis-Hastings (Q341139)

The Metropolis-Hastings (MH) algorithm is a Markov chain Monte Carlo (MCMC) method of profound importance to many fields of mathematics such as Bayesian inference and statistical mechanics. In the present paper, an application of the non-reversible Metropolis-Hastings (NRMH) method in continuous setting by developing the necessary theory and applying, as first examples, the theory of Gaussian distribution in three and nine dimensions is provided. The empirical autocorrelation and estimated asymptotic variance for NRMH applied to these examples show significant improvement compared to MH with identical step-size. There exist two basic approaches to the construction of non-reversible chains from reversible chains. In the present paper, the approach of the non-reversibility without altering the state space is constructed. The MH is extended to NRMH which allows non-reversible transition. In Section 2, the notion of vorticity matrix is recalled. The non-reversible Markov chains are given in the terms of the vorticity. In Subsection 2.3, the concept of the non-reversible Hastings algorithm to construct Markov chains is discussed. In Section 3, the advantages of non-reversible Markov chains in finite state are briefly discussed. It is shown how to transform a discrete time Markov chain into a continuous time chain. In Section 4, it is explained how to extend the idea of non-reversible MH algorithm to an Euclidean state space. In Subsection 4.2, the use of the Langevin diffusions for simulating from a target density and the potential role of NRMH in within are discussed. The concept of the Euler-Maruyama discretization of the Langevin diffusion is developed. In Subsection 4.3, the NRMH method is applied to sampling multivariate Gaussian distributions. In Subsection 4.4, the concept of the empirical autocorrelation function (EACF) is given. The EACF for 3-dimensional and 9-dimensional examples are considered. The performance of NRMH is compared to MH with identical step-size. In Section 5, the efficiency of non-reversible Markov chains is discussed. In Appendix 1, the NRMH algorithm in general state spaces is considered and Theorem 4.1 is proved. In Appendix 2, Theorem 4.2 is proved.