A potential reset mechanism for the modulation of decision processes under uncertainty

Abstract

Humans and other mammals flexibly select actions in noisy, uncertain contexts, quickly using feedback to adapt their decision policies to either explore other options or to exploit what they know. Drawing inspiration from the plasticity of cortico-basal ganglia-thalamic circuitry, we recently developed a cognitive model of decision-making that uses both a value-driven learning signal to update an internal estimate of state action-value (i.e., conflict in the probability of reward between two choices) and a change-point-driven learning signal that adapts to changes in reward contingencies (i.e., a previously high value target becoming devalued). In this work, we expand on previous results from our group (Bond, Dunovan, & Verstynen, 2018) to more carefully detail how these environmental signals drive changes in the decision process. Across nine separate behavioral testing sessions, we independently manipulated the level of value-conflict and volatility in action-outcome contingencies. Using a hierarchical drift diffusion model, we found that the belief in the value difference between options had the greatest influence on decision processes, impacting drift rate, while estimates of environmental change had a smaller, but detectable influence on the decision threshold. Taken together, these findings bolster our previous work showing how separate environmental signals impact different aspects of the decision algorithm.