The use of an OSCE with a standardized patient actor to assess professionalism and communication for the oral certifying exam: the initial experience of the American Board of Urology.

Abstract

Purpose: Promotion of professionalism/communication (P/C) is a strategic initiative within the American Board of Medical Specialties; however, reliable assessment of this competency in the certification process is lacking. Herein we present the findings of the American Board of Urology's (ABU) P/C Objective Structured Clinical Exam (OSCE) which was implemented on the 2023 and 2024 certifying exams.

Methods: The certifying exam was administered to 694 candidates (335 in 2023 and 359 in 2024). Each candidate was administered two 5-item OSCEs and four 10-item Standard Oral Examinations (SOEs). One OSCE focused on P/C with simulated patient (SP) actors and one on diagnosis/imaging. SP actors interacted with examinees on the 10-minute P/C OSCE. A criterion-referenced standard was used for pass/fail decisions and the Rasch model was used for scoring.

Results: The candidate's mean score and protocol difficulty did not differ significantly from 2023 to 2024. Reliability with the P/C and diagnostic OSCEs was similar both years. In 2023, the diagnostic OSCE had a higher average score than the P/C OSCE with both OSCEs scoring higher than the SOEs. In 2024, the average diagnostic OSCE score was higher than the P/C OSCE and SOEs which had similar scores. There was low correlation between the OSCEs and SOEs both years.

Conclusion: The initial experience of P/C OSCEs on the ABU certification examination showed acceptable scoring, reliability, and low correlation with standard protocols. Initial data suggests that the P/C OSCE is a distinct construct from SOEs which specifically assesses P/C skills in the oral certification process.