Measuring Residents' Competence in Chest Tube Insertion on Thiel-Embalmed Bodies: A Validity Study.

Abstract

Introduction: Chest tube insertions (CTIs) have a high complication rate, prompting the training of technical skills in simulated settings. However, assessment tools require validity evidence prior to their implementation. This study aimed to collect validity evidence for assessment of technical skills in CTI on Thiel-embalmed human bodies.

Methods: Invitations were sent to residents and staff from the departments of surgery, pulmonology, and emergency medicine. Participants were familiarized with the Thiel body and the supplied equipment. Standardized clinical context and instructions were provided. All participants performed 2 CTIs and were assessed with the Assessment for Competence in Chest Tube InsertiON (ACTION) tool, consisting of a 17-item rating scale and a 16-item error checklist. Live and post hoc video-based assessments by 2 raters were performed. Generalizability analysis was performed to evaluate reliability. Mean scores and errors were compared using a mixed-model repeated measures analysis of variance (ANOVA). A pass/fail score was determined using the contrasting groups' method.

Results: Ten novices and 8 experienced participants completed the study. The Generalizability coefficients were moderate for the rating scale (0.75), and low for the error checklist (0.4). Novices scored lower on the rating scale?? (44±6.7/68 vs 50.8 ± 5.7/68, P = 0.024), but did not commit significantly more errors (1.6 ± 1.1/16 vs 1.0 ± 0.6/16, P = 0.066). A pass/fail score of 47/68 was established.

Conclusion: The rating scale in the Assessment for Competence in Chest Tube InsertiON tool has a robust validity argument for use on Thiel-embalmed bodies, allowing it to be used in simulation-based mastery learning curricula. In contrast, its error checklist has insufficient reliability and validity to be used for summative assessment.