Performance-based assessment in a pre-clinical medical school chest radiology curriculum: Student achievement and attitudes

Abstract

Objective: The purpose of this study was to evaluate student achievement and attitudes pertaining to a performance-based assessment in a medical student chest radiology curriculum using mixed quantitative and qualitative methods. Methods: One-hundred-and-sixty-one participating second-year students, in the 2015-16 academic year took a post-curriculum multiple choice question (MCQ) exam. Students also underwent a performance-based assessment in the form of a chest x-ray interpretation small-group session administered by radiology faculty at a picture archiving and communications system (PACS) workstation. Each student verbally interpreted one chest radiograph showing one of 6 pathologies and was given a numerical rating based on a standardized rubric. This score was compared to the correctness of the student’s answer to the corresponding MCQ question on the same topic. All students completed a post-session questionnaire. Open-ended free-text responses regarding student attitudes were coded into qualitative themes by three independent raters. High inter-rater agreement was demonstrated by average agreement index of .82 or greater (ranging from 0-1, with 1 indicating perfect agreement) for responses to the most frequent themes. Results: There was no significant association between scores on MCQ exam and performance-based assessment. Up to 90% of post-session questionnaire respondents indicated a Likert rating of 5 (strongly agree) when asked if the sessions improved understanding of chest x-rays and their ability to identify specific radiologic pathology. Dominant themes from open-ended responses were then derived. Conclusions: Lack of agreement between student performance on the chest x-ray interpretation small-group session and that on the parallel MCQ exam suggests that they each measure a different type of achievement, with the former emphasizing skills over knowledge. Features most commonly valued by students in the performance-based assessment were its ability to reinforce prior knowledge, supply an authentic and relevant PACS experience simulating real life, and provide an opportunity for active practice of radiology interpretive skills.