Medical Teacher最新文献

Background: Manually creating multiple-choice questions (MCQ) is inefficient. Automatic item generation (AIG) offers a scalable solution, with two main approaches: template-based and non-template-based (AI-driven). Template-based AIG ensures accuracy but requires significant expert input to develop templates. In contrast, AI-driven AIG can generate questions quickly but with inaccuracies. The Hybrid AIG combines the strengths of both methods. However, neither have MCQs been generated using the Hybrid AIG approach nor has any validity evidence been provided.

Methods: We generated MCQs using the Hybrid AIG approach and investigated the validity evidence of these questions by determining whether experts could identify the correct answers. We used a custom ChatGPT to develop an item template, which were then fed into Gazitor, a template-based AIG (non-AI) software. A panel of medical doctors identified the answers.

Results: Of 105 decisions, 101 (96.2%) matched the software's correct answer. In all MCQs (100%), the experts reached a consensus on the correct answer. The evidence corresponds to the 'Relations to Other Variables' in Messick's validity framework.

Conclusions: The Hybrid AIG approach can enhance the efficiency of MCQ generation while maintaining accuracy. It mitigates concerns about hallucinations while benefiting from AI.

Purpose: Accreditation is a key regulatory strategy to ensure high-quality medical education occurs across the world's growing number of medical schools. There is general agreement on standards for medical schools, but no consensus on the evidence needed to show standards are being met. This study characterized the evidence solicited from schools by World Federation for Medical Education (WFME)-recognized agencies.

Materials and methods: We collected documents from WFME-recognized agencies as of July 2022. We aligned self-evaluation guidance with relevant WFME standards, then compared common features of evaluation guidance across agencies.

Results: We obtained 75 documents (2460 pages) from 21 of 28 WFME-recognized agencies, which collectively accredited 690 programs in 57 independent states covering a population of over 1.8 billion people. Self-evaluation guidance varied by the amount of information provided by an agency, structure for reporting, and data types requested from schools. Overall, requirements ranged from completing a database hundreds of pages long to drafting a maximum 50-page report outlining strengths and weaknesses.

Conclusions: Evidence requested from schools to demonstrate they met accreditation standards varied widely. This has implications for the reliability of accreditation judgments and resources required in school self-evaluations to ensure accreditation processes are aligned with the needs of learners, educators, and the public.

Purpose: Training of medical students, interns and residents (junior professionals, JP) happens predominantly through workplace learning. This is supported by medical specialists (MS) and varies by MS, workplace, medical specialty, and social context. This study aimed to gain insights into the facilitation of workplace learning in clinical practice by MS.

Method: Thirteen semi-structured interviews were conducted with MS at the gynaecology and paediatrics department in an academic and three non-academic hospitals in the Netherlands. Participants were purposively sampled to be involved in the training of JP. Transcribed interviews were thematically analysed to describe facilitators and barriers for workplace learning.

Results: Workplace learning was interpreted as a broad concept, with some MS acknowledging its occurrence beyond created opportunities. Four themes revealed facilitators and barriers related to (1) Connection and alignment, (2) Attitude, (3) Organizational and learning dynamics, and (4) Policy and Logistics. Our research suggests MS' beliefs and awareness regarding workplace learning guide their ability to recognize and utilize learning moments.

Conclusions: Our findings underscore the importance of a work environment for MS that is founded on interpersonal relationships, supportive logistics of healthcare institutions and a culture of active involvement within the team, all of which are essential for recognizing and utilizing learning opportunities.

Purpose: The latest reform of French medical studies has moved the National Ranking Examination before residency to the beginning of the sixth-year for undergraduate medical students, thus placing unprecedented workload during the preceding summer. The main objective was to determine whether study conditions and psychosocial factors were associated with student success in this model of intense workload.

Materials and methods: An online survey designed with six student-partners was sent at a French Medical School after the examination in 2023. The primary outcome was student success in achieving their main goal (Ranking, Knowledge, Well-being). A machine-learning model (eXtreme Gradient Boosting) was developed and explained using Artificial Intelligence. An AI-guided multivariate logistic regression was performed, Odd Ratios were calculated.

Results: Out of 123 responses, 75 (61%) of the students achieved their main goal. Motivation and socialization during the summer were the two most important variables for predicting student success. In guided multivariate logistic regression, summer motivation (Odd Ratio = 4.12, 95%CI[1.75-10.30]), summer loneliness (Odd Ratio = 0.35, 95%CI[0.14-0.86]), and student's main goal (Ranking, Odd Ratio = 2.94, 95%CI[1.15-7.79]) were associated with student success.

Conclusions: Motivation and socialization during the summer preceding high-stakes examinations are strongly predictive of undergraduate medical students' success. This study highlights the importance of well-being during summer for student success.[Box: see text].

Background: The Objective Structured Clinical Examination (OSCE) is a cornerstone of medical education that uses a structured approach to assess clinical skills and competency. A well-designed checklist is essential to enhance the validity of OSCE exams. This study aimed to determine whether a clinically discriminatory checklist (CDC) improves the validity of the OSCE compared with an assessment using the thoroughness checklist (TC), with a particular focus on clinical reasoning.

Methods: Fourteen OSCE case scenarios with both TC and CDC were developed. Each case was administered to 350-1170 fourth-year medical students in nine medical schools within the Seoul-Gyeonggi-area (Korea) during their OSCEs in 2019 and 2020. We also conducted interstation examinations after standardized patient encounters to assess clinical reasoning ability. The validities of OSCE scores based on the TCs and CDCs were compared.

Results: The OSCE using a CDC (rather than a TC) enabled better item discrimination but provided a lower internal consistency coefficient and worse standard measurement error. Clinical reasoning scores derived using patient notes were significantly correlated with OSCE scores but varied according to the characteristics of each case, indicating that OSCE scores derived using CDCs did not assess clinical reasoning ability more accurately than OSCE scores obtained using TCs.

Conclusions: This study found that using a CDC to limit checklist items did not improve OSCE validity and did not reflect clinical reasoning ability. Further development of robust assessment strategies that support and evaluate clinical reasoning abilities is needed.