Scoring story recall for individual differences research: Central details, peripheral details, and automated scoring.

Abstract

Story recall is an episodic memory paradigm that is popular among researchers interested in the effects of aging, disease, and/or injury on memory functioning; it is less popular among individual-differences researchers studying neurotypical young adults. One reason differential psychologists may favor other episodic memory paradigms is that the prospect of scoring story recall is daunting, as it typically requires manually scoring hundreds or thousands of freely recalled narratives. In this study, I investigated two questions related to scoring story recall for individual differences research. First, whether there is anything to gain by scoring story recall for memory of central and peripheral details or if a single score is sufficient. Second, I investigated whether scoring can be automated using computational methods - namely, BERTScore and GPT-4. A total of 235 individuals participated in this study. At the latent variable level, central and peripheral factors were highly correlated (r = .99), and the two factors correlated with external factors (viz., fluid intelligence, crystallized intelligence, and working memory capacity) similarly. Regarding automated scoring, both BERTScore and GPT-4 derived scores were strongly correlated with manually derived scores (r ≥ .97); additionally, factors estimated from the various scoring methods all showed a similar pattern of correlations with the external factors. Thus, differential psychologists may be able to streamline scoring by disregarding detail type and by using automated approaches. Further research is needed, particularly of the automated approaches, as both BERTScore and GPT-4 derived scores were occasionally leptokurtic while manual scores were not.