Journal of Educational Evaluation for Health Professions最新文献

Clinical reasoning is a core competency in medical practice; however, deficits in this domain among residents are often difficult to identify and remediate because of its cognitive complexity and the absence of standardized assessment approaches. This scoping review aimed to map and analyze existing evidence on strategies to remediate clinical reasoning skill deficits in underperforming medical residents. Using the Arksey and O'Malley framework as refined by Levac and his colleagues, and reported in accordance with PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines, we systematically searched PubMed, Scopus, MEDLINE, Web of Science, SpringerLink, ProQuest, and EBSCOhost for studies published between 2000 and 2024. Definitions of clinical reasoning, underperformance, and remediation were adopted from prior literature. Twenty studies met the inclusion criteria, comprising original research and literature reviews in multiple medical specialties. Methods for identifying clinical reasoning deficits included written, oral, and performance-based assessments, as well as routine workplace-based evaluations. Remediation strategies ranged from structured institutional programs to individualized, case-specific interventions, with coaching, deliberate practice, guided reflection, and structured thinking frameworks frequently employed. Two studies reported positive outcomes following completion of remediation for clinical reasoning deficits. Key enablers included psychological safety, learner engagement, and accessible faculty support, whereas barriers included learner resistance, inadequate baseline knowledge, faculty skill limitations, and institutional resource constraints. Effective remediation requires early identification, comprehensive diagnostic assessment, and tailored, coaching-based interventions supported by institutional commitment. Nonetheless, substantial variability in definitions, remediation protocols, and evaluation methods highlights the need for greater standardization and further research across diverse contexts to inform evidence-based frameworks for clinical reasoning remediation.

Reference management software (RMS) represents a cornerstone of modern academic writing and publishing. For decades, programs such as EndNote, Zotero, and Mendeley have played central roles in facilitating citation organization, bibliography formatting, and collaborative scholarship. Although each platform has introduced unique innovations, persistent limitations remain, particularly with respect to usability, accessibility, and accuracy. In parallel, the rise of generative artificial intelligence has introduced an unprecedented challenge: the inadvertent inclusion of fabricated or incorrect references mistakenly incorporated into manuscripts. This phenomenon has exposed a critical limitation of traditional RMS platforms, namely their inability to verify reference authenticity. Against this backdrop, new solutions have emerged. One such example is CiteWell (https://citewell.org/), an artificial intelligence (AI)-era RMS that introduces several notable innovations, including PubMed-integrated verification, an intuitive interface for new users, customizable journal-specific styles, and multilingual accessibility. This review provides a comprehensive historical overview of RMS, evaluates the strengths and weaknesses of major platforms, and positions emerging AI-based tools as a new paradigm that combines traditional reference management with essential safeguards for contemporary academic challenges.

Purpose: Artificial intelligence (AI)-driven simulation is an emerging approach in healthcare education that enhances learning effectiveness. This review examined its impact on the development of non-technical skills among medical learners.

Methods: Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a systematic review was conducted using the following databases: Web of Science, ScienceDirect, Scopus, and PubMed. The quality of the included studies was assessed using the Mixed.

Methods: Appraisal Tool. The protocol was previously registered in PROSPERO (CRD420251038024).

Results: Of the 1,442 studies identified in the initial search, 20 met the inclusion criteria, involving 2,535 participants. The simulators varied considerably, ranging from platforms built on symbolic AI methods to social robots powered by computational AI. Among the 15 AI-driven simulators, 10 used ChatGPT or its variants as virtual patients. Several studies evaluated multiple non-technical skills simultaneously. Communication and clinical reasoning were the most frequently assessed skills, appearing in 12 and 6 studies, respectively, which generally reported positive outcomes. Improvements were also noted in decision-making, empathy, self-confidence, critical thinking, and problem-solving. In contrast, emotional regulation, assessed in a single study, showed no significant difference. Notably, none of the studies examined reflection, reflective practice, teamwork, or leadership.

Conclusion: AI-driven simulation shows substantial potential for enhancing non-technical skills in medical education, particularly communication and clinical reasoning. However, its effects on several other non-technical skills remain unclear. Given heterogeneity in study designs and outcome measures, these findings should be interpreted cautiously. These considerations highlight the need for further research to support integrating this innovative approach into medical curricula.

Purpose: This study aimed to evaluate the feasibility of general-purpose large language models (LLMs) in addressing inequities in medical licensure exam preparation for Thailand’s National Medical Licensing Examination (ThaiNLE), which currently lacks standardized public study materials.

Methods: We assessed 4 multi-modal LLMs (GPT-4, Claude 3 Opus, Gemini 1.0/1.5 Pro) using a 304-question ThaiNLE Step 1 mock examination (10.2% image-based), applying deterministic API configurations and 5 inference repetitions per model. Performance was measured via micro- and macro-accuracy metrics compared against historical passing thresholds.

Results: All models exceeded passing scores, with GPT-4 achieving the highest accuracy (88.9%; 95% confidence interval, 88.7–89.1), surpassing Thailand’s national average by more than 2 standard deviations. Claude 3.5 Sonnet (80.1%) and Gemini 1.5 Pro (72.8%) followed hierarchically. Models demonstrated robustness across 17 of 20 medical domains, but variability was noted in genetics (74.0%) and cardiovascular topics (58.3%). While models demonstrated proficiency with images (Gemini 1.0 Pro: +9.9% vs. text), text-only accuracy remained superior (GPT4o: 90.0% vs. 82.6%).

Conclusion: General-purpose LLMs show promise as equitable preparatory tools for ThaiNLE Step 1. However, domain-specific knowledge gaps and inconsistent multi-modal integration warrant refinement before clinical deployment.

The peer review process ensures the integrity of scientific research. This is particularly important in the medical field, where research findings directly impact patient care. However, the rapid growth of publications has strained reviewers, causing delays and potential declines in quality. Generative artificial intelligence, especially large language models (LLMs) such as ChatGPT, may assist researchers with efficient, high-quality reviews. This review explores the integration of LLMs into peer review, highlighting their strengths in linguistic tasks and challenges in assessing scientific validity, particularly in clinical medicine. Key points for integration include initial screening, reviewer matching, feedback support, and language review. However, implementing LLMs for these purposes will necessitate addressing biases, privacy concerns, and data confidentiality. We recommend using LLMs as complementary tools under clear guidelines to support, not replace, human expertise in maintaining rigorous peer review standards.

This study aims to evaluate whether incorporating live consultation observations into interprofessional education (IPE) improves learning evaluation scores among healthcare professionals and students. A retrospective cross-sectional analysis was conducted using evaluation data from AHP IPE sessions held from January 2020 to December 2023 across 7 primary care clinics in Singapore. Evaluation scores were compared between sessions with facilitated discussions only (n=667) and sessions with additional live consultation observations (n=501). Logistic regression was used to analyze factors associated with perfect evaluation scores. Sessions that included live consultations were significantly more likely to achieve perfect evaluation scores (odds ratio [OR], 1.68; 95% confidence interval [CI], 1.27-2.22). Nursing/care coordinator and allied health professions (OR 2.07 and 1.76 respectively) were significantly more likely to give perfect scores compared to medical professions. Healthcare professionals were also more likely to give perfect scores than students (OR, 1.52; 95% CI,1.08-2.14), indicating enhanced perceived effectiveness. These findings support the use of experiential learning strategies to optimize interprofessional training outcomes.

Purpose: Interprofessional education (IPE) is increasingly recognized as critical to preparing health professionals for collaborative practice, yet rigorous assessment remains limited by a lack of psychometrically sound instruments. Building on a previously developed questionnaire for physicians, this study aimed to expand the scale to include allied health professionals and to evaluate whether the factor structure remained consistent across professions. We hypothesized that a similar factor structure would emerge from the combined dataset, thereby supporting the scale's generalizability.

Methods: This observational study included 930 healthcare professionals in the United States (379 physicians, 419 nurses, 76 pharmacists, and others) who completed a 35-item questionnaire addressing IPE competency domains. Data were collected between December 2019 and May 2020. Exploratory factor analysis was employed to examine the factor structure, followed by item response theory (IRT) analyses to assess item fit, reliability, and validity. Raw data are available upon request.

Results: Factor analysis of 22 retained items confirmed a 5-factor solution: teamwork and communication, patient-centered care, roles and responsibilities, ethics and attitudes, and reflective practice, explaining 59% of the variance. Subscale reliabilities ranged from α=0.65 to 0.87. IRT analyses supported construct validity and measurement precision, while identifying areas for refinement in reflective practice.

Conclusion: This study demonstrates that the scale is reliable, valid, and generalizable across diverse health professions. It provides a robust tool for assessing attitudes toward IPE, offering value for curriculum evaluation, institutional benchmarking, and future longitudinal research on professional identity formation and collaborative practice.