Point-of-Care-Ultrasound Quality Assurance Data From Fellow-Performed Exams in a Pediatric Emergency Department

Objectives

Point-of-care-ultrasound (POCUS) is increasingly used by pediatric emergency medicine (PEM) fellows, but scant data exists on the accuracy of exam interpretations. Our goal was to determine whether agreement on exam interpretation between quality assurance (QA) faculty (reference standard) and PEM fellows varied by fellowship year or exam type.

Methods

Retrospective review of fellow-performed POCUS exams between January 2019 and June 2022. Negative binomial (NB) random effects regression was used to account for longitudinal measurement of individual fellow performance across 3 years. Fixed effects were exam type and fellowship year. To assess between- and within-user variability across time, a random intercept and slope were included for each fellow.

Results

Exactly 3032 exams, performed by 24 fellows, were included. Raw proportion agreement by fellowship year was high for all exam types (≥88%). From the NB model, there was no statistically significant effect of fellowship year on the mean count of agreement. The relative risk (RR) of agreement for exam types was greatest for cardiac vs other types. The standard deviations for the random intercept and random slope were 0.09 and 0.04, respectively, with a correlation of −0.94.

Conclusions

PEM fellows generally interpret exams correctly, with little variation through fellowship, although those who began with more basic skills showed more progress over time. Fellowship year did not influence the likelihood of correct interpretation but there was variation across exam type, with the best agreement for cardiac exams. The extent to which disagreements between fellows and QA faculty represent clinically significant errors requires further study.