Perceptions of the Emergency Medicine Resident Selection Process by Program Directors Following the Transition to a Pass/Fail USMLE Step 1.

Abstract

Background: Beginning January 26th, 2022, the National Board of Medical Examiners transitioned scoring of the United States Medical Licensing Examination (USMLE) Step 1 from a 3-digit score to pass/fail. In the past, the Step 1 score has been weighted heavily by program directors (PDs) as one of the most important metrics when assessing medical student's competitiveness.

Objective: The objective of this study was to evaluate the perceptions of emergency medicine (EM) PDs on the transition to a pass/fail USMLE Step 1 exam, and to elicit the opinions of EM PDs on the USMLE examinations' ability to predict resident performance.

Methods: A survey consisting of ranking and multiple-choice questions was sent to EM PDs. The multiple-choice questions were asked to determine EM PDs level of confidence in the ability of Step 1 and Step 2 Clinical Knowledge (CK) to predict a student's ability to succeed in residency. The ranking questions focused on assessing each program's current resident selection practices in comparison to expected selection criteria changes following a transition to pass/fail Step 1. R studio and MATLAB were used for statistical analysis, and a P value <0.05 was considered significant.

Results: The survey was completed by 57 (20.21%) EM PDs. When asked if Step 1 and Step 2 CK are accurate predictors of a resident's ability to perform clinically within EM, only 10.5% of PDs answered 'yes' to Step 1 being predictive, compared to 31.6% for Step 2 CK. Regarding selection criteria, the top quartile of attributes (standardized letters of evaluation [1st], away rotations [2nd], clerkship grades [3rd] and Step 2 CK score [4th]) remained the same following the transition.

Conclusion: Our results indicate that the top quartile of attributes might remain the same, despite most PDs agreeing that Step 2 CK is a better predictor of a resident's performance.