Adaptive learning rate in dynamical binary environments: the signature of adaptive information processing.

Abstract

Adaptive mechanisms of learning models play critical roles in interpreting adaptive behavior of humans and animals. Different learning models, varying from Bayesian models, deep learning or regression models to reward-based reinforcement learning models, adopt similar update rules. These update rules can be reduced to the same generalized mathematical form: the Rescorla-Wagner equation. In this paper, we construct a hierarchical Bayesian model with an adaptive learning rate for inferring a hidden probability in a dynamical binary environment, and analysis the adaptive behavior of the model on synthetic data. The update rule of the model state turns out to be an extension of the Rescorla-Wagner equation. The adaptive learning rate is modulated by beliefs and environment uncertainty. Our results underscore adaptive learning rate as mechanistic component in efficient and accurate inference, as well as the signature of information processing in adaptive machine learning models.