Academic Medicine最新文献

Purpose: Blinding in pediatric residency recruitment and the influence of implicit biases have not been formally studied. This study examined whether blinding to race and/or gender influences the selection of candidates for pediatric residency interviews and assessed the role of respondent implicit bias.

Method: An electronic survey was sent to all U.S. pediatric residency program directors in spring 2023. Nonresponders were sent weekly reminders for 5 weeks (survey remained open for 6 weeks). Respondents rated 5 fictitious applicants, each randomly assigned a gender (male, female, or blinded) and race (Black, White, or blinded), and completed an Implicit Association Test (IAT) to assess unconscious attitudes about race. The survey then asked about current strategies to mitigate unconscious bias in residency recruitment.

Results: Responses were received from 85 of 202 programs (42%). All 85 program leaders reported using implicit bias training, with 64 of 83 (77%) using blinding and 74 of 84 (88%) using standardized rubrics to score applications as strategies to mitigate bias. The IAT revealed no statistically significant difference in the proportion of respondents with a positive implicit attitude toward Black versus White people (W = 840, P = .20). Statistically significant main effects were found for applicant race and interaction between applicant race and respondent IAT score, with respondents rating applicants with unknown race lower by a mean (95% CI) of 0.61 (0.07-1.16) points on the 5-point scale than the same applicants presenting as White or Black ( t222 = 2.2, P = .03) and respondents rating White or unknown race applicants lower when their implicit attitudes toward Black people were more positive ( t207 = -4.0, P < .001 and t208 = -2.9, P = .004, respectively).

Conclusions: Blinding applicant race may adversely impact some applicants' interview prospects, suggesting that caution be applied when considering blinding to address implicit bias.

Problem: Assessing students in competency-based medical education can be time-consuming and demanding for faculty, especially with large classes and complex topics. Traditional methods can lead to inconsistencies and a lack of targeted feedback. Innovative and accessible solutions to improve the efficiency, objectivity, and effectiveness of assessment in medical education are needed.

Approach: From September 2024 to February 2025, the authors piloted the use of large language models (LLMs) with retrieval-augmented generation to assess students' understanding of moral injury. The authors selected and uploaded 6 seminal articles on moral injury within military and veteran populations to Google Gemini 1.5 Pro. They tasked the same LLM with creating a grading rubric based on these articles to assess 165 student responses in a military medical ethics course (Uniformed Services University of the Health Sciences). The authors uploaded both the generated rubric and the student responses to each of 3 LLMs (Google Gemini 1.5 Pro, Google Gemini 2.0 Flash, and OpenAI ChatGPT-4o) with a prompt to generate scores for the student responses.

Outcomes: In the authors' expert opinion, an LLM (Google Gemini 1.5 Pro) successfully generated a grading rubric that captured the nuances of moral injury and its implications for military medical practice. The LLMs' scoring accuracy was compared against 2 experienced educators to generate validity evidence. The best-performing model, OpenAI ChatGPT-4o, demonstrated an interrater reliability of 0.77 and 0.68 for reviewers 1 and 2, respectively, indicating a higher level of agreement between the LLM and both individual reviewers than between the 2 reviewers (0.57).

Next steps: While this approach shows promise, faculty oversight is necessary to ensure ethical accountability and address potential biases. Further research is needed to optimize the integration of AI and human capabilities in assessment to ultimately enhance the quality of health care professional education and improve patient outcomes.