The orbitofrontal cortex as a negative feedback control system: computational modeling and fMRI

Abstract

In this work we address two inter-related issues. First, the computational roles of the orbitofrontal cortex (OFC) and hippocampus in value-based decision-making have been unclear, with various proposed roles in value representation, cognitive maps, and prospection. Second, reinforcement learning models have been slow to adapt to more general problems in which the reward values of states may change over time, thus requiring different Q values for a given state at different times. We have developed a model of artificial general intelligence that treats much of the brain as a high dimensional control system in the framework of control theory. We show with computational modeling and combined fMRI and representational similarity analysis (RSA) that the model can autonomously learn to solve problems and provides a clear computational account of how a number of brain regions, particularly the OFC, interact to guide behavior to achieve arbitrary goals.