Modeling Cognitive-Affective Processes With Appraisal and Reinforcement Learning

Abstract

Computational models can advance affective science by shedding light onto the interplay between cognition and emotion from an information processing point of view. We propose a computational model of emotion that integrates reinforcement learning (RL) and appraisal theory, establishing a formal relationship between reward processing, goal-directed task learning, cognitive appraisal, and emotional experiences. The model achieves this by formalizing four evaluative checks from the component process model (CPM) in terms of temporal difference learning updates: suddenness, goal relevance, goal conduciveness, and power. The formalism is task independent and can be applied to any task that is represented as a Markov decision problem (MDP) and solved using RL. We evaluate the model by predicting a range of human emotions based on a series of vignette studies, highlighting its potential to improve our understanding of the role of reward processing in affective experiences.