Echo state network-enhanced symbolic regression for spatio-temporal binary stochastic cellular automata

Abstract

Binary spatio-temporal data are common in many application areas. Such data can be considered from many perspectives, including via deterministic or stochastic cellular automata (CA), where local rules govern the transition probabilities that describe the evolution of the 0 and 1 states across space and time. One implementation of a stochastic CA for such data is via a spatio-temporal generalized linear model (or mixed model), with the local rule covariates being included in the transformed mean response. However, in many applications we do have a complete understanding of the local rules and must instead explore the rules space, which can be accomplished through symbolic regression. Even with a learned rule space, the data-driven rules may be insufficient to describe the process behavior and it is helpful to augment the transformed linear predictor with a latent spatio-temporal dynamic process. Here, we demonstrate for the first time that an echo state network (ESN) latent process can be used to enhance symbolic regression-learned local rule covariates. We implement this in a hierarchical Bayesian framework with regularized horseshoe priors on the ESN output weight matrices, which extends the ESN literature as well. Finally, we gain added expressiveness from the ESNs by considering an ensemble of ESN reservoirs, which we accommodate through weighted model averaging, which is also new to the ESN literature. We demonstrate our methodology on a simulated process in which we assume we do not know all of the local CA rules, as well as on multiple environmental data sets.