Pub Date : 2024-09-01Epub Date: 2024-06-12DOI: 10.1097/ACM.0000000000005784
Tanios Dagher, Julie Oyler, Valerie G Press, Tia Kostas, Joyce Tang, Robert Hight, Mark Saathoff, Marie McKinnon, Jenna Ford Jackson, Michael McGinty, Korry Schwanz, Jeanne Farnan, Vineet Arora, Irsk Anderson
Problem: Medical school graduates enter a complex health care delivery system involving interprofessional teamwork and multifaceted value-based patient care decisions. However, current curricula on health systems science (HSS) are piecemeal, lecture based, and confined to preclinical training.
Approach: The VISTA program is a longitudinal, immersive learning curriculum integrated into the University of Chicago Pritzker School of Medicine curriculum between 2016 and 2018. Key components include a unit-based nursing interprofessional team experience, a discharge objective structured clinical examination (OSCE), a patient safety simulation, and the implementation of a Choosing Wisely SmartPhrase. Graduates before 2016-2017 and after 2018-2020 VISTA implementation completed a Likert-style survey assessing attitudes, knowledge, and behaviors on HSS topics. A free response question solicited improvement areas. The Association of American Medical Colleges (AAMC) Graduation Questionnaire (GQ) was also examined.
Outcomes: The overall VISTA survey response rate was 59%, with 126 fourth-year medical student respondents before VISTA and 120 after VISTA. Compared with pre-VISTA graduates, post-VISTA graduates reported a significantly higher rate of competence on the HSS questions, with the greatest increases seen in effective communication at discharge (n = 73/126 [57.9%] to 116/120 [96.7%], P < .001), knowledge on safety event reporting (n = 53/126 [42.1%] to 96/120 [79.8%], P < .001), and considering costs in making health care decisions (n = 76/126 [60.3%] to 117/120 [97.5%], P < .001). All were directly addressed through experiential learning interventions, and 2 were intended practice behaviors. VISTA graduate responses to free-text questions demonstrated a more nuanced understanding of HSS compared with pre-VISTA responses. The AAMC GQ data showed increased agreement with an item that mapped to HSS understanding.
Next steps: The VISTA program provides a model for institutions to enhance HSS education between curricular overhauls. Next steps include implementing value-added roles and additional immersive learning exercises.
{"title":"Development and Implementation of an Experiential Longitudinal Health Systems Science Thread Into an Existing Medical School Curriculum.","authors":"Tanios Dagher, Julie Oyler, Valerie G Press, Tia Kostas, Joyce Tang, Robert Hight, Mark Saathoff, Marie McKinnon, Jenna Ford Jackson, Michael McGinty, Korry Schwanz, Jeanne Farnan, Vineet Arora, Irsk Anderson","doi":"10.1097/ACM.0000000000005784","DOIUrl":"10.1097/ACM.0000000000005784","url":null,"abstract":"<p><strong>Problem: </strong>Medical school graduates enter a complex health care delivery system involving interprofessional teamwork and multifaceted value-based patient care decisions. However, current curricula on health systems science (HSS) are piecemeal, lecture based, and confined to preclinical training.</p><p><strong>Approach: </strong>The VISTA program is a longitudinal, immersive learning curriculum integrated into the University of Chicago Pritzker School of Medicine curriculum between 2016 and 2018. Key components include a unit-based nursing interprofessional team experience, a discharge objective structured clinical examination (OSCE), a patient safety simulation, and the implementation of a Choosing Wisely SmartPhrase. Graduates before 2016-2017 and after 2018-2020 VISTA implementation completed a Likert-style survey assessing attitudes, knowledge, and behaviors on HSS topics. A free response question solicited improvement areas. The Association of American Medical Colleges (AAMC) Graduation Questionnaire (GQ) was also examined.</p><p><strong>Outcomes: </strong>The overall VISTA survey response rate was 59%, with 126 fourth-year medical student respondents before VISTA and 120 after VISTA. Compared with pre-VISTA graduates, post-VISTA graduates reported a significantly higher rate of competence on the HSS questions, with the greatest increases seen in effective communication at discharge (n = 73/126 [57.9%] to 116/120 [96.7%], P < .001), knowledge on safety event reporting (n = 53/126 [42.1%] to 96/120 [79.8%], P < .001), and considering costs in making health care decisions (n = 76/126 [60.3%] to 117/120 [97.5%], P < .001). All were directly addressed through experiential learning interventions, and 2 were intended practice behaviors. VISTA graduate responses to free-text questions demonstrated a more nuanced understanding of HSS compared with pre-VISTA responses. The AAMC GQ data showed increased agreement with an item that mapped to HSS understanding.</p><p><strong>Next steps: </strong>The VISTA program provides a model for institutions to enhance HSS education between curricular overhauls. Next steps include implementing value-added roles and additional immersive learning exercises.</p>","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"971-975"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141312214","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-06-17DOI: 10.1097/ACM.0000000000005790
Mark Ehioghae, Nana Danso, Pinky Jha
{"title":"Lessons Learned From a Mentorship Platform for Underrepresented Minority Medical Students.","authors":"Mark Ehioghae, Nana Danso, Pinky Jha","doi":"10.1097/ACM.0000000000005790","DOIUrl":"10.1097/ACM.0000000000005790","url":null,"abstract":"","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":"99 9","pages":"938-939"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142082500","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-06-12DOI: 10.1097/ACM.0000000000005787
Gabriella Schmuter, Robert A Beale
{"title":"Engaging Learners With the Utility of Electronic Medical Record Templates in Patient Note Writing.","authors":"Gabriella Schmuter, Robert A Beale","doi":"10.1097/ACM.0000000000005787","DOIUrl":"10.1097/ACM.0000000000005787","url":null,"abstract":"","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"940"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141312215","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-07-29DOI: 10.1097/ACM.0000000000005749
David A Cook, Christopher R Stephenson
Purpose: Learner engagement is the energy learners exert to remain focused and motivated to learn. The Learner Engagement Instrument (LEI) was developed to measure learner engagement in a short continuing professional development (CPD) activity. The authors validated LEI scores using validity evidence of internal structure and relationships with other variables.
Method: Participants attended 1 of 4 CPD courses (1 in-person, 2 online livestreamed, and 1 either in-person or livestreamed) in 2018, 2020, 2021, and 2022. Confirmatory factor analysis was used to examine model fit for several alternative structural models, separately for each course. The authors also conducted a generalizability study to estimate score reliability. Associations were evaluated between LEI scores and Continuing Medical Education Teaching Effectiveness (CMETE) scores and participant demographics. Statistical methods accounted for repeated measures by participants.
Results: Four hundred fifteen unique participants attended 203 different CPD presentations and completed the LEI 11,567 times. The originally hypothesized 4-domain model of learner engagement (domains: emotional, behavioral, cognitive in-class, cognitive out-of-class) demonstrated best model fit in all 4 courses, with comparative fit index ≥ 0.99, standardized root mean square residual ≤ 0.031, and root mean square error of approximation ≤ 0.047. The reliability for overall scores and domain scores were all acceptable (50-rater G-coefficient ≥ 0.74) except for the cognitive in-class domain (50-rater G-coefficient of 0.55 to 0.66). Findings were similar for both in-person and online delivery modalities. Correlation of LEI scores with teaching effectiveness was confirmed (rho=0.58), and a small correlation was found with participant age (rho=0.19); other associations were small and not statistically significant. Using these findings, we generated a shortened 4-item instrument, the LEI Short Form.
Conclusions: This study confirms a 4-domain model of learner engagement and provides validity evidence that supports using LEI scores to measure learner engagement in both in-person and livestreamed CPD activities.
目的:学习者投入度是指学习者为保持学习的专注性和积极性而付出的精力。学习者参与度工具(LEI)是为了测量学习者在短期持续专业发展(CPD)活动中的参与度而开发的。作者利用内部结构的有效性证据以及与其他变量的关系对 LEI 分数进行了验证:参与者参加了 2018 年、2020 年、2021 年和 2022 年的 4 门 CPD 课程(1 门面授课程、2 门在线直播课程和 1 门面授或在线课程)中的 1 门。作者分别对每门课程进行了确认性因子分析,以检验几种可供选择的结构模型的模型拟合度。作者还进行了一项可推广性研究,以估算分数的可靠性。还评估了 LEI 分数与继续医学教育教学效果 (CMETE) 分数和学员人口统计学之间的关联。所有统计方法都考虑了参与者的重复测量:415名参与者参加了203场不同的继续医学教育讲座,完成了11,567次LEI。最初假设的学习者参与度 4 领域模型(领域:情感、行为、课内认知、课外认知)在所有 4 门课程中均表现出最佳模型拟合度,比较拟合指数≥0.99,标准化均方根残差≤0.031,均方根近似误差≤0.047。除课堂认知领域(50 人 G 系数为 0.55 至 0.66)外,总分和领域分的信度均可接受(50 人 G 系数≥ 0.74)。所有结果在面授和在线授课模式下都相似。LEI 分数与教学效果的相关性得到了证实(rho 0.58),与参与者年龄的相关性较小(rho 0.19);其他相关性较小,且无统计学意义。根据这些研究结果,我们制作了一个简短的 4 个项目的工具,即 LEI 简表:本研究证实了学习者参与度的 4 领域模型,并提供了有效性证据,支持使用 LEI 分数来衡量学习者在现场和直播 CPD 活动中的参与度。
{"title":"Validation of the Learner Engagement Instrument for Continuing Professional Development.","authors":"David A Cook, Christopher R Stephenson","doi":"10.1097/ACM.0000000000005749","DOIUrl":"10.1097/ACM.0000000000005749","url":null,"abstract":"<p><strong>Purpose: </strong>Learner engagement is the energy learners exert to remain focused and motivated to learn. The Learner Engagement Instrument (LEI) was developed to measure learner engagement in a short continuing professional development (CPD) activity. The authors validated LEI scores using validity evidence of internal structure and relationships with other variables.</p><p><strong>Method: </strong>Participants attended 1 of 4 CPD courses (1 in-person, 2 online livestreamed, and 1 either in-person or livestreamed) in 2018, 2020, 2021, and 2022. Confirmatory factor analysis was used to examine model fit for several alternative structural models, separately for each course. The authors also conducted a generalizability study to estimate score reliability. Associations were evaluated between LEI scores and Continuing Medical Education Teaching Effectiveness (CMETE) scores and participant demographics. Statistical methods accounted for repeated measures by participants.</p><p><strong>Results: </strong>Four hundred fifteen unique participants attended 203 different CPD presentations and completed the LEI 11,567 times. The originally hypothesized 4-domain model of learner engagement (domains: emotional, behavioral, cognitive in-class, cognitive out-of-class) demonstrated best model fit in all 4 courses, with comparative fit index ≥ 0.99, standardized root mean square residual ≤ 0.031, and root mean square error of approximation ≤ 0.047. The reliability for overall scores and domain scores were all acceptable (50-rater G-coefficient ≥ 0.74) except for the cognitive in-class domain (50-rater G-coefficient of 0.55 to 0.66). Findings were similar for both in-person and online delivery modalities. Correlation of LEI scores with teaching effectiveness was confirmed (rho=0.58), and a small correlation was found with participant age (rho=0.19); other associations were small and not statistically significant. Using these findings, we generated a shortened 4-item instrument, the LEI Short Form.</p><p><strong>Conclusions: </strong>This study confirms a 4-domain model of learner engagement and provides validity evidence that supports using LEI scores to measure learner engagement in both in-person and livestreamed CPD activities.</p>","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"1024-1031"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140873570","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-05-15DOI: 10.1097/ACM.0000000000005769
Christopher R Runyon, Miguel A Paniagua, Francine A Rosenthal, Andrea L Veneziano, Lauren McNaughton, Constance T Murray, Polina Harik
Problem: Many non-workplace-based assessments do not provide good evidence of a learner's problem representation or ability to provide a rationale for a clinical decision they have made. Exceptions include assessment formats that require resource-intensive administration and scoring. This article reports on research efforts toward building a scalable non-workplace-based assessment format that was specifically developed to capture evidence of a learner's ability to justify a clinical decision.
Approach: The authors developed a 2-step item format called SHARP (SHort Answer, Rationale Provision), referring to the 2 tasks that comprise the item. In collaboration with physician-educators, the authors integrated short-answer questions into a patient medical record-based item starting in October 2021 and arrived at an innovative item format in December 2021. In this format, a test-taker interprets patient medical record data to make a clinical decision, types in their response, and pinpoints medical record details that justify their answers. In January 2022, a total of 177 fourth-year medical students, representing 20 U.S. medical schools, completed 35 SHARP items in a proof-of-concept study.
Outcomes: Primary outcomes were item timing, difficulty, reliability, and scoring ease. There was substantial variability in item difficulty, with the average item answered correctly by 44% of students (range, 4%-76%). The estimated reliability (Cronbach α ) of the set of SHARP items was 0.76 (95% confidence interval, 0.70-0.80). Item scoring is fully automated, minimizing resource requirements.
Next steps: A larger study is planned to gather additional validity evidence about the item format. This study will allow comparisons between performance on SHARP items and other examinations, examination of group differences in performance, and possible use cases for formative assessment. Cognitive interviews are also planned to better understand the thought processes of medical students as they work through the SHARP items.
{"title":"SHARP (SHort Answer, Rationale Provision): A New Item Format to Assess Clinical Reasoning.","authors":"Christopher R Runyon, Miguel A Paniagua, Francine A Rosenthal, Andrea L Veneziano, Lauren McNaughton, Constance T Murray, Polina Harik","doi":"10.1097/ACM.0000000000005769","DOIUrl":"10.1097/ACM.0000000000005769","url":null,"abstract":"<p><strong>Problem: </strong>Many non-workplace-based assessments do not provide good evidence of a learner's problem representation or ability to provide a rationale for a clinical decision they have made. Exceptions include assessment formats that require resource-intensive administration and scoring. This article reports on research efforts toward building a scalable non-workplace-based assessment format that was specifically developed to capture evidence of a learner's ability to justify a clinical decision.</p><p><strong>Approach: </strong>The authors developed a 2-step item format called SHARP (SHort Answer, Rationale Provision), referring to the 2 tasks that comprise the item. In collaboration with physician-educators, the authors integrated short-answer questions into a patient medical record-based item starting in October 2021 and arrived at an innovative item format in December 2021. In this format, a test-taker interprets patient medical record data to make a clinical decision, types in their response, and pinpoints medical record details that justify their answers. In January 2022, a total of 177 fourth-year medical students, representing 20 U.S. medical schools, completed 35 SHARP items in a proof-of-concept study.</p><p><strong>Outcomes: </strong>Primary outcomes were item timing, difficulty, reliability, and scoring ease. There was substantial variability in item difficulty, with the average item answered correctly by 44% of students (range, 4%-76%). The estimated reliability (Cronbach α ) of the set of SHARP items was 0.76 (95% confidence interval, 0.70-0.80). Item scoring is fully automated, minimizing resource requirements.</p><p><strong>Next steps: </strong>A larger study is planned to gather additional validity evidence about the item format. This study will allow comparisons between performance on SHARP items and other examinations, examination of group differences in performance, and possible use cases for formative assessment. Cognitive interviews are also planned to better understand the thought processes of medical students as they work through the SHARP items.</p>","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"976-980"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140960618","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-05-01DOI: 10.1097/ACM.0000000000005747
Capri P Alex, H Barrett Fromme, Larrie Greenberg, Michael S Ryan, Sarah Gustafson, Maya K Neeley, Shaughn Nunez, Molly E Rideout, Jessica VanNostrand, Nicola M Orlov
Purpose: Direct observation (DO) enables assessment of vital competencies, such as clinical skills. Despite national requirement that medical students experience DOs during each clerkship, the frequency, length, quality, and context of these DOs are not well established. This study examines the quality, quantity, and characteristics of DOs obtained during pediatrics clerkships across multiple institutions.
Method: This multimethod study was performed at 6 U.S.-based institutions from March to October 2022. In the qualitative phase, focus groups and/or semistructured interviews were conducted with third-year medical students at the conclusion of pediatrics clerkships. In the quantitative phase, the authors administered an internally developed instrument after focus group discussions or interviews. Qualitative data were analyzed using thematic analysis, and quantitative data were analyzed using anonymous survey responses.
Results: Seventy-three medical students participated in 20 focus groups, and 71 (97.3%) completed the survey. The authors identified 7 themes that were organized into key principles: before, during, and after DO. Most students reported their DOs were conducted primarily by residents (62 [87.3%]) rather than attendings (6 [8.4%]) in inpatient settings. Participants reported daily attending observation of clinical reasoning (38 [53.5%]), communication (39 [54.9%]), and presentation skills (58 [81.7%]). One-third reported they were never observed taking a history by an inpatient attending (23 [32.4%]), and one-quarter reported they were never observed performing a physical exam (18 [25.4%]).
Conclusions: This study revealed that students are not being assessed for performing vital clinical skills in the inpatient setting by attendings as frequently as previously believed. When observers set expectations, create a safe learning environment, and follow up with actionable feedback, medical students perceive the experience as valuable; however, the DO experience is currently suboptimal. Therefore, a high-quality, competency-based clinical education for medical students is necessary to directly drive future patient care by way of a competent physician workforce.
{"title":"Exploring Medical Student Experiences With Direct Observation During the Pediatric Clerkship.","authors":"Capri P Alex, H Barrett Fromme, Larrie Greenberg, Michael S Ryan, Sarah Gustafson, Maya K Neeley, Shaughn Nunez, Molly E Rideout, Jessica VanNostrand, Nicola M Orlov","doi":"10.1097/ACM.0000000000005747","DOIUrl":"10.1097/ACM.0000000000005747","url":null,"abstract":"<p><strong>Purpose: </strong>Direct observation (DO) enables assessment of vital competencies, such as clinical skills. Despite national requirement that medical students experience DOs during each clerkship, the frequency, length, quality, and context of these DOs are not well established. This study examines the quality, quantity, and characteristics of DOs obtained during pediatrics clerkships across multiple institutions.</p><p><strong>Method: </strong>This multimethod study was performed at 6 U.S.-based institutions from March to October 2022. In the qualitative phase, focus groups and/or semistructured interviews were conducted with third-year medical students at the conclusion of pediatrics clerkships. In the quantitative phase, the authors administered an internally developed instrument after focus group discussions or interviews. Qualitative data were analyzed using thematic analysis, and quantitative data were analyzed using anonymous survey responses.</p><p><strong>Results: </strong>Seventy-three medical students participated in 20 focus groups, and 71 (97.3%) completed the survey. The authors identified 7 themes that were organized into key principles: before, during, and after DO. Most students reported their DOs were conducted primarily by residents (62 [87.3%]) rather than attendings (6 [8.4%]) in inpatient settings. Participants reported daily attending observation of clinical reasoning (38 [53.5%]), communication (39 [54.9%]), and presentation skills (58 [81.7%]). One-third reported they were never observed taking a history by an inpatient attending (23 [32.4%]), and one-quarter reported they were never observed performing a physical exam (18 [25.4%]).</p><p><strong>Conclusions: </strong>This study revealed that students are not being assessed for performing vital clinical skills in the inpatient setting by attendings as frequently as previously believed. When observers set expectations, create a safe learning environment, and follow up with actionable feedback, medical students perceive the experience as valuable; however, the DO experience is currently suboptimal. Therefore, a high-quality, competency-based clinical education for medical students is necessary to directly drive future patient care by way of a competent physician workforce.</p>","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"997-1006"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140861330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-03-14DOI: 10.1097/ACM.0000000000005690
Paulina Perez Mejias, Gustavo Lara, Alex Duran, Rashelle Musci, Nancy A Hueppchen, Roy C Ziegelstein, Pamela A Lipsett
Purpose: To determine whether students' self-reported race/ethnicity and sex were associated with grades earned in 7 core clerkships. A person-centered approach was used to group students based on observed clerkship grade patterns. Predictors of group membership and predictive bias by race/ethnicity and sex were investigated.
Method: Using data from 6 medical student cohorts at Johns Hopkins University School of Medicine (JHUSOM), latent class analysis was used to classify students based on clerkship grades. Multinomial logistic regression was employed to investigate if preclerkship measures and student demographic characteristics predicted clerkship performance-level groups. Marginal effects for United States Medical Licensing Exam (USMLE) Step 1 scores were obtained to assess the predictive validity of the test on group membership by race/ethnicity and sex. Predictive bias was examined by comparing multinomial logistic regression prediction errors across racial/ethnic groups.
Results: Three clerkship performance-level groups emerged from the data: low, middle, and high. Significant predictors of group membership were race/ethnicity, sex, and USMLE Step 1 scores. Black or African American students were more likely (odds ratio [OR] = 4.26) to be low performers than White students. Black or African American (OR = 0.08) and Asian students (OR = 0.41) were less likely to be high performers than White students. Female students (OR = 2.51) were more likely to be high performers than male students. Patterns of prediction errors observed across racial/ethnic groups showed predictive bias when using USMLE Step 1 scores to predict clerkship performance-level groups.
Conclusions: Disparities in clerkship grades associated with race/ethnicity were found among JHUSOM students, which persisted after controlling for USMLE Step 1 scores, sex, and other preclerkship performance measures. Differential predictive validity of USMLE Step 1 exam scores and systematic error predictions by race/ethnicity show predictive bias when using USMLE Step 1 scores to predict clerkship performance across racial/ethnic groups.
{"title":"Disparities in Medical School Clerkship Grades Associated With Sex, Race, and Ethnicity: A Person-Centered Approach.","authors":"Paulina Perez Mejias, Gustavo Lara, Alex Duran, Rashelle Musci, Nancy A Hueppchen, Roy C Ziegelstein, Pamela A Lipsett","doi":"10.1097/ACM.0000000000005690","DOIUrl":"10.1097/ACM.0000000000005690","url":null,"abstract":"<p><strong>Purpose: </strong>To determine whether students' self-reported race/ethnicity and sex were associated with grades earned in 7 core clerkships. A person-centered approach was used to group students based on observed clerkship grade patterns. Predictors of group membership and predictive bias by race/ethnicity and sex were investigated.</p><p><strong>Method: </strong>Using data from 6 medical student cohorts at Johns Hopkins University School of Medicine (JHUSOM), latent class analysis was used to classify students based on clerkship grades. Multinomial logistic regression was employed to investigate if preclerkship measures and student demographic characteristics predicted clerkship performance-level groups. Marginal effects for United States Medical Licensing Exam (USMLE) Step 1 scores were obtained to assess the predictive validity of the test on group membership by race/ethnicity and sex. Predictive bias was examined by comparing multinomial logistic regression prediction errors across racial/ethnic groups.</p><p><strong>Results: </strong>Three clerkship performance-level groups emerged from the data: low, middle, and high. Significant predictors of group membership were race/ethnicity, sex, and USMLE Step 1 scores. Black or African American students were more likely (odds ratio [OR] = 4.26) to be low performers than White students. Black or African American (OR = 0.08) and Asian students (OR = 0.41) were less likely to be high performers than White students. Female students (OR = 2.51) were more likely to be high performers than male students. Patterns of prediction errors observed across racial/ethnic groups showed predictive bias when using USMLE Step 1 scores to predict clerkship performance-level groups.</p><p><strong>Conclusions: </strong>Disparities in clerkship grades associated with race/ethnicity were found among JHUSOM students, which persisted after controlling for USMLE Step 1 scores, sex, and other preclerkship performance measures. Differential predictive validity of USMLE Step 1 exam scores and systematic error predictions by race/ethnicity show predictive bias when using USMLE Step 1 scores to predict clerkship performance across racial/ethnic groups.</p>","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"1007-1015"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"140137492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-06-26DOI: 10.1097/ACM.0000000000005793
Gustavo A Patino, Laura Weiss Roberts
{"title":"The Need for Greater Transparency in Journal Submissions That Report Novel Machine Learning Models in Health Professions Education.","authors":"Gustavo A Patino, Laura Weiss Roberts","doi":"10.1097/ACM.0000000000005793","DOIUrl":"10.1097/ACM.0000000000005793","url":null,"abstract":"","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"935-937"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141460561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2023-05-25DOI: 10.1097/ACM.0000000000005280
Ross Gay
{"title":"To the Fig Tree on 9th and Christian.","authors":"Ross Gay","doi":"10.1097/ACM.0000000000005280","DOIUrl":"https://doi.org/10.1097/ACM.0000000000005280","url":null,"abstract":"","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":"99 9","pages":"964"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142300184","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-09-01Epub Date: 2024-06-12DOI: 10.1097/ACM.0000000000005785
Nam S Danny Hoang
{"title":"Protecting and Learning From LGBTQ Students.","authors":"Nam S Danny Hoang","doi":"10.1097/ACM.0000000000005785","DOIUrl":"10.1097/ACM.0000000000005785","url":null,"abstract":"","PeriodicalId":50929,"journal":{"name":"Academic Medicine","volume":" ","pages":"1015"},"PeriodicalIF":5.3,"publicationDate":"2024-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141312218","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"教育学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}