Recent experiments and theories of human decision-making suggest positive and negative errors are processed and encoded differently by serotonin and dopamine, with serotonin possibly serving to oppose dopamine and protect against risky decisions. We introduce a temporal difference (TD) model of human decision-making to account for these features. Our model involves two critics, an optimistic learning system and a pessimistic learning system, whose predictions are integrated in time to control how potential decisions compete to be selected. Our model predicts that human decision-making can be decomposed along two dimensions: the degree to which the individual is sensitive to (1) risk and (2) uncertainty. In addition, we demonstrate that the model can learn about the mean and standard deviation of rewards, and provide information about reaction time despite not modeling these variables directly. Lastly, we simulate a recent experiment to show how updates of the two learning systems could relate to dopamine and serotonin transients, thereby providing a mathematical formalism to serotonin's hypothesized role as an opponent to dopamine. This new model should be useful for future experiments on human decision-making.
Psychopathic traits and the childhood analogue, callous-unemotional traits, have been severely neglected by the research field in terms of mechanistic, falsifiable accounts. This is surprising given that some of the core symptoms of the disorder point towards problems with basic components of associative learning. In this manuscript we describe a new mechanistic account that is concordant with current cognitive theories of psychopathic traits and is also able to replicate previous empirical data. The mechanism we describe is one of individual differences in an index we have called, "learning window width". Here we show how variation in this index would result in different outcome expectations which, in turn, would lead to differences in behaviour. The proposed mechanism is intuitive and simple with easily calculated behavioural implications. Our hope is that this model will stimulate discussion and the use of mechanistic and computational accounts to improve our understanding in this area of research.
Action selection requires a policy that maps states of the world to a distribution over actions. The amount of memory needed to specify the policy (the policy complexity) increases with the state-dependence of the policy. If there is a capacity limit for policy complexity, then there will also be a trade-off between reward and complexity, since some reward will need to be sacrificed in order to satisfy the capacity constraint. This paper empirically characterizes the trade-off between reward and complexity for both schizophrenia patients and healthy controls. Schizophrenia patients adopt lower complexity policies on average, and these policies are more strongly biased away from the optimal reward-complexity trade-off curve compared to healthy controls. However, healthy controls are also biased away from the optimal trade-off curve, and both groups appear to lie on the same empirical trade-off curve. We explain these findings using a cost-sensitive actor-critic model. Our empirical and theoretical results shed new light on cognitive effort abnormalities in schizophrenia.
Positive self-beliefs are important for well-being, and are influenced by how others evaluate us during social interactions. Mechanistic accounts of self-beliefs have mostly relied on associative learning models. These account for choice behaviour but not for the explicit beliefs that trouble socially anxious patients. Neither do they speak to self-schemas, which underpin vulnerability according to psychological research. Here, we compared belief-based and associative computational models of social-evaluation, in individuals that varied in fear of negative evaluation (FNE), a core symptom of social anxiety. We used a novel analytic approach, 'clinically informed model-fitting', to determine the influence of FNE symptom scores on model parameters. We found that high-FNE participants learn more easily from negative feedback about themselves, manifesting in greater self-negative learning rates. Crucially, we provide evidence that this bias is underpinned by an overall reduced belief about self-positive attributes. The study population could be characterized equally well by belief-based or associative models, however large individual differences in model likelihood indicated that some individuals relied more on an associative (model-free), while others more on a belief-guided strategy. Our findings have therapeutic importance, as positive belief activation may be used to specifically modulate learning.
Author summary: Understanding how we form and maintain positive self-beliefs is crucial to understanding how things go awry in disorders such as social anxiety. The loss of positive self-belief in social anxiety, especially in inter-personal contexts, is thought to be related to how we integrate evaluative information that we receive from others. We frame this social information integration as a learning problem and ask how people learn whether someone approves of them or not. We thus elucidate why the decrease in positive evaluations manifests only for the self, but not for an unknown other, given the same information. We investigated the mechanics of this learning using a novel computational modelling approach, comparing models that treat the learning process as series of stimulusresponse associations with models that treat learning as updating of beliefs about the self (or another). We show that both models characterise the process well and that individuals higher in symptoms of social anxiety learn more from negative information specifically about the self. Crucially, we provide evidence that this originates from a reduction in the amount of positive attributes that are activated when the individual is placed in a social evaluative context.
We provide a proof of principle for an evolutionary systems theory (EST) of depression. This theory suggests that normative depressive symptoms counter socioenvironmental volatility by increasing interpersonal support via social signalling and that this response depends upon the encoding of uncertainty about social contingencies, which can be targeted by neuromodulatory antidepressants. We simulated agents that committed to a series of decisions in a social two-arm bandit task before and after social adversity, which precipitated depressive symptoms. Responses to social adversity were modelled under various combinations of social support and pharmacotherapy. The normative depressive phenotype responded positively to social support and simulated treatments with antidepressants. Attracting social support and administering antidepressants also alleviated anhedonia and social withdrawal, speaking to improvements in interpersonal relationships. These results support the EST of depression by demonstrating that following adversity, normative depressed mood preserved social inclusion with appropriate interpersonal support or pharmacotherapy.
Affective bias - a propensity to focus on negative information at the expense of positive information - is a core feature of many mental health problems. However, it can be caused by wide range of possible underlying cognitive mechanisms. Here we illustrate this by focusing on one particular behavioural signature of affective bias - increased tendency of anxious/depressed individuals to predict lower rewards - in the context of the Signal Detection Theory (SDT) modelling framework. Specifically, we show how to apply this framework to measure affective bias and compare it to the behaviour of an optimal observer. We also show how to extend the framework to make predictions about bias when the individual holds incorrect assumptions about the decision context. Building on this theoretical foundation, we propose five experiments to test five hypothetical sources of this affective bias: beliefs about prior probabilities, beliefs about performance, subjective value of reward, learning differences, and need for accuracy differences. We argue that greater precision about the mechanisms driving affective bias may eventually enable us to better understand the mechanisms underlying mood and anxiety disorders.