Evaluating script concordance tests (SCTs) through the lens of Bayesian reasoning: Enhancing assessment in medical education.

Abstract

Background: Script Concordance Tests (SCTs) represent an innovative assessment method which have been introduced in the 2024 French National Ranking Examinations (EDN). These tests compare a student's clinical reasoning with that of a panel of experts under conditions of uncertainty. Typically, the question involves the impact of new information on an initially proposed hypothesis, with answers given on a Likert scale.

Main findings: This article aims to didactically illustrate how SCTs are consistent with probabilistic reasoning as modeled by Bayes' theorem. In addition, by comparing SCT writing guidelines with Bayesian reasoning concepts, several ambiguities were identified: (1) What stage of clinical reasoning do SCTs evaluate? (2) What are the appropriate labels for Likert scale responses? (3) Does the expert panel provide a relevant reference for SCTs?

Conclusions: Currently, many of these questions remain unanswered in the literature, with recent data suggesting that experienced physicians' responses to SCTs are often biased. Beyond their use as an assessment tool in the EDN, SCTs offer a valuable opportunity to develop and deepen the teaching of probabilistic reasoning in medical education and serve as a potential area of research to improve clinical practice.