A Memory-Augmented Reinforcement Learning Model of Food Caching Behaviour in Birds

Birds of the crow family are well known for their complex cognition. In laboratory experiments it has been observed that jays adapt food caching strategies to anticipated needs and rely on a memory of the what, where and when of previous caching events for cache recovery. While this behaviour is well studied, little is known about the algorithms and neural processes that produce this behaviour. We present a computational model and propose a neural implementation of food caching behaviour. Our model features latent hunger variables for motivational control, an associative memory for snapshots of the sensory states during caching events, a system memory consolidation for flexible decoding of the age of a memory, a stimulus-driven retrieval mechanism, and rewardmodulated update of retrieval and caching policies during inspection of caches. We show that our model is in quantitative agreement with the results of 22 behavioural experiments. Our methodology of a formalization of experimental protocols via a domain-specific language is transferable to other domains and may serve as a tool to design new experiments and foster collaboration between experimentalists and theoreticians. Our model is an example of a structured reinforcement learning algorithm that could have evolved in species that operate in partially observable environments.