The Use of an Objective Structured Clinical Examination With a Standardized Patient Actor to Assess Professionalism and Communication for the Oral Certifying Examination: The Initial Experience of the American Board of Urology.

Abstract

Introduction: Promotion of professionalism/communication (P/C) is a strategic initiative within the American Board of Medical Specialties; however, reliable assessment of this competency in the certification process is lacking. In this article, we present the findings of the American Board of Urology's P/C objective structured clinical examination (OSCE), which was implemented on the 2023 and 2024 certifying examinations.

Methods: The certifying examination was administered to 694 candidates (335 in 2023 and 359 in 2024). Each candidate was administered two 5-item OSCEs and four 10-item standard oral examinations (SOEs). One OSCE focused on P/C with simulated patient (SP) actors and one on diagnosis/imaging. SP actors interacted with examinees on the 10-minute P/C OSCE. A criterion-referenced standard was used for pass/fail decisions, and the Rasch model was used for scoring.

Results: The candidate's mean score and protocol difficulty did not differ significantly from 2023 to 2024. Reliability with the P/C and diagnostic OSCEs was similar both years. In 2023, the diagnostic OSCE had a higher average score than the P/C OSCE, with both OSCEs scoring higher than the SOEs. In 2024, the average diagnostic OSCE score was higher than that of the P/C OSCE and SOEs, which had similar scores. There was low correlation between the OSCEs and SOEs both years.

Conclusions: The initial experience of P/C OSCEs on the American Board of Urology certification examination showed acceptable scoring, reliability, and low correlation with standard protocols. Initial data suggest that the P/C OSCE is a distinct construct from SOEs that specifically assesses P/C skills in the oral certification process.