Creating Affective Autonomous Characters Using Planning in Partially Observable Stochastic Domains

The ability to reason about and respond to their own emotional states can enhance the believability of Non-Player Characters (NPCs). In this paper, we use a Partially Observable Markov Decision Process (POMDP)-based framework to model emotion over time. A two-level appraisal model, involving quick and reactive vs. slow and deliberate appraisals, is proposed for the creation of affective autonomous characters based on POMDPs, wherein the probability of goal satisfaction is used in an appraisal and reappraisal process for emotion generation. We not only extend Probabilistic Computation Tree Logic (PCTL) for reasoning about the properties of emotional states based on POMDPs but also illustrate how four reactive (primary) emotions and nine deliberate (secondary) emotions can be derived by combining PCTL with the belief-desire theory of emotion. The results of an empirical study suggest that the proposed model can be used to create characters that appear to be more believable and more intelligent.