Transforming Medical Education: Leveraging Large Language Models to Enhance PBL - A Proof-of-Concept Study.

Abstract

The alignment of learning materials with the learning objectives (LOs) is critical for successfully implementing the Problem-Based Learning (PBL) curriculum. This study investigated the capabilities of Gemini Advanced, a large language model (LLM), in creating clinical vignettes that align with LOs, and comprehensive tutor guides. This study used a faculty-written clinical vignette about diabetes mellitus for third-year medical students. We submitted the LOs and the associated clinical vignette and tutor guide to the LLM to evaluate their alignment and generate new versions. Four faculty members compared both versions using a structured questionnaire. The mean evaluation scores for original and LLM-generated versions are reported. The LLM identified new triggers for the clinical vignette to align it better with the LOs. Moreover, it restructured the tutor guide for better organization and flow and included thought-provoking questions. The medical information provided by the LLM was scientifically appropriate and accurate. LLM-generated clinical vignette scored higher (3.0 vs. 1.25) for the alignment with the LOs. However, the original version scored better for being educational-level appropriate (2.25 vs. 1.25) and adhering to PBL design (2.50 vs. 1.25). The LLM-generated tutor guide scored higher for better flow (3.0 vs. 1.25), comprehensive and relevant content (2.75 vs. 1.50) and thought-provoking questions (2.25 vs. 1.75). However, LLM-generated learning material lacked visual elements. In conclusion, this study demonstrated that Gemini could align and improve PBL learning materials. By leveraging the potential of LLMs while acknowledging their limitations, medical educators can create innovative and effective learning experiences for future physicians.