Examiner-examinee gender concordance does not impact ratings on the American Board of Surgery Vascular Surgery Certifying Examination

Objective

Implicit bias is a potential factor in the severity of examinee rating during oral examinations. Ratings may be impacted by examinee characteristics, such as gender, that are independent of examinee knowledge base, clinical judgment, or test-taking ability. The effects of examiner-examinee gender concordance in the Vascular Surgery Certifying Examination (VCE) have not been previously studied. We explored whether examiner ratings and likelihood of passing the examination were influenced by gender concordance among examiners and examinees.

Methods

Data collected from examinees who first attempted the VCE between 2018 and 2023 were analyzed. There were 1005 examinees (69.3% male and 30.1% female) and 121 examiners (71.9% male, and 28.1% female). Linear mixed-effects models and generalized linear mixed-effects models were used to evaluate the effects of examinee and examiner gender on VCE ratings and likelihood of passing the examination.

Results

Examiner-examinee gender concordance had no significant impact on examiner ratings or likelihood of passing the examination. In addition, examinee gender alone had no significant impact on VCE rating or pass rates. Only Vascular Qualifying Examination scores explained more than 1% of the variance in total VCE scores for the gender model (F(1, 1003.5) = 71.08, P < .01, R² = 3%). Vascular Qualifying Examination scores were positively related to total VCE scores.

Conclusions

Although implicit bias has the potential to impact examiner scoring, there is no evidence that this is the case with respect to gender in the VCE of the American Board of Surgery.