Fast copula variational inference

Abstract

ABSTRACT Mean-field variational inference, built on fully factorisations, can be efficiently solved; however, it ignores the dependencies between latent variables, resulting in lower performance. To address this, the copula variational inference (CVI) method is proposed by using the well-established copulas to effectively capture posterior dependencies, leading to better approximations. However, it suffers from a computational issue, where the optimisation for big models with massive latent variables is quite time-consuming. This is mainly caused by the expensive sampling when forming noisy Monte Carlo gradients in CVI. For CVI speedup, in this paper we propose a novel fast CVI (abbr. FCVI). In FCVI, we derive the gradient of CVI objective by an expectation of the mean-field factorisation. Therefore, we can achieve a much efficient sampling from the -dimensional mean-field factorisation, enabling to reduce the sampling complexity from to . To evaluate FCVI, we compare it against baseline methods on modelling performance and runtime. Experimental results demonstrate that FCVI is on a par with CVI, but runs much faster.