Exact computation of Transfer Entropy with Path Weight Sampling

Abstract

Information processing in networks entails a dynamical transfer of information between stochastic variables. Transfer entropy is widely used for quantification of the directional transfer of information between input and output trajectories. However, currently there is no exact technique to quantify transfer entropy given the dynamical model of a general network. Here we introduce an exact computational algorithm, Transfer Entropy-Path Weight Sampling (TE-PWS), to quantify transfer entropy and its variants in an arbitrary network in the presence of multiple hidden variables, nonlinearity, transient conditions, and feedback. TE-PWS extends a recently introduced algorithm Path Weight Sampling (PWS) and uses techniques from the statistical physics of polymers and trajectory sampling. We apply TE-PWS to linear and nonlinear systems to reveal how transfer entropy can overcome naive applications of data processing inequalities in presence of feedback.