Adaptive Stereographic MCMC

Abstract

In order to tackle the problem of sampling from heavy tailed, high dimensional distributions via Markov Chain Monte Carlo (MCMC) methods, Yang, Latuszy\'nski, and Roberts (2022) (arXiv:2205.12112) introduces the stereographic projection as a tool to compactify $\mathbb{R}^d$ and transform the problem into sampling from a density on the unit sphere $\mathbb{S}^d$. However, the improvement in algorithmic efficiency, as well as the computational cost of the implementation, are still significantly impacted by the parameters used in this transformation. To address this, we introduce adaptive versions of the Stereographic Random Walk (SRW), the Stereographic Slice Sampler (SSS), and the Stereographic Bouncy Particle Sampler (SBPS), which automatically update the parameters of the algorithms as the run progresses. The adaptive setup allows us to better exploit the power of the stereographic projection, even when the target distribution is neither centered nor homogeneous. We present a simulation study showing each algorithm's robustness to starting far from the mean in heavy tailed, high dimensional settings, as opposed to Hamiltonian Monte Carlo (HMC). We establish a novel framework for proving convergence of adaptive MCMC algorithms over collections of simultaneously uniformly ergodic Markov operators, including continuous time processes. This framework allows us to prove LLNs and a CLT for our adaptive Stereographic algorithms.