Convergence of stratified MCMC sampling of non-reversible dynamics

Abstract

We present a form of stratified MCMC algorithm built with non-reversible stochastic dynamics in mind. It can also be viewed as a generalization of the exact milestoning method or form of NEUS. We prove the convergence of the method under certain assumptions, with expressions for the convergence rate in terms of the process’s behavior within each stratum and large-scale behavior between strata. We show that the algorithm has a unique fixed point which corresponds to the invariant measure of the process without stratification. We will show how the convergence speeds of two versions of the algorithm, one with an extra eigenvalue problem step and one without, related to the mixing rate of a discrete process on the strata, and the mixing probability of the process being sampled within each stratum. The eigenvalue problem version also relates to local and global perturbation results of discrete Markov chains, such as those given by Van Koten, Weare et. al.