Journal of the American Medical Informatics Association最新文献

Objective: This study aims to compare the effectiveness of 2 ambient AI scribe technologies in reducing physician burnout, improving workflow satisfaction, and enhancing documentation efficiency through a randomized crossover trial.

Materials and methods: An open-label randomized crossover trial involving 160 outpatient clinicians was conducted at a tertiary academic medical center. Volunteers were randomized to 2 groups of 80 with 2 crossover periods. We assessed workflow satisfaction (1-7 scale), burnout (Copenhagen Burnout Index), and efficiency metrics (eg, electronic health record time outside scheduled hours, documentation time, etc.). Data was analyzed using Wilcoxon signed-rank tests and generalized linear mixed models.

Results: Surveys from 136 respondents were analyzed. Clinicians reported greater improvements in satisfaction with product B (2.51 points on a 7-point scale) compared to product A (1.91 points; mean difference: 0.60, 95% CI: 0.32-0.90). Both tools reduced personal and work burnout scores, but differences between tools were not meaningful. Product B demonstrated greater reductions in average minutes-in-notes per day compared to product A (B - A = -3.19 minutes; 95% CI -4.87 to -1.50). No meaningful differences were observed in pajama time or patient-related burnout.

Discussion: Both tools improved workflow satisfaction and reduced burnout, with product B showing superior performance in satisfaction and documentation time. However, efficiency metrics like pajama time were largely unaffected, potentially due to participant selection bias and the study period's timing.

Conclusion: Product B yielded greater satisfaction and time savings compared to product A, though both tools effectively reduced physician burnout and improved workflow satisfaction.

Objectives: To explore the complexities of eliminating race correction in clinical artificial intelligence (AI), the pitfalls of naive solutions, and to propose systematic strategies for equitable model development.

Background and significance: Race correction in clinical AI, as in traditional medicine, introduces biases with potentially harmful consequences. Simple removal of race from models is insufficient due to the lasting influence of historically biased data.

Approach: We analyze 4 standardized scenarios to demonstrate how race correction manifests in clinical AI: use of race-corrected variables, explicit inclusion of race, inference via proxy variables, and use of race-specific models.

Results: For each scenario, the intuitive solution to removing race correction fails to eliminate bias, often due to legacy effects embedded in the data. More thoughtful approaches are required.

Discussion: Ending race correction in clinical AI requires deliberate, context-sensitive interventions, inclusion of diverse stakeholders, and strategies to make model reasoning more transparent and auditable.

Objectives: Accurate triage in emergency departments (ED) is critical for appropriate resource allocation. While artificial intelligence (AI) has been explored for triage, prior models relied on summarized clinical scenarios. We aimed to develop and evaluate large language models (LLMs) trained on real-world clinical conversations to classify patient urgency.

Materials and methods: We used a nationally curated dataset of anonymized triage-level conversations from 3 tertiary Korean hospitals. Two BERT-based models were developed to classify urgency per the Korean Triage and Acuity Scale (KTAS) into urgent (KTAS 3) or non-urgent (KTAS 4-5). One model tokenized the entire conversation, while the other applied a hierarchical structure with sentence-level tokenization and speaker-role embeddings. Performance metrics included accuracy, precision, recall, and F1-score. We compared our models against ChatGPT GPT-4o and ClinicalBERT, and assessed explainability using SHapley Additive exPlanations (SHAP).

Results: A total of 5244 clinical conversations, 1057 triage-level dialogues were used, with 950 for training and 107 for testing. Our model with hierarchical structure achieved accuracies of 75.94%, significantly outperforming ChatGPT (56.68%) or fine-tuned ClinicalBERT (69.42%). For urgent cases, the best model achieved a recall of 0.9610, outperforming ChatGPT (0.5352). SHapley Additive exPlanations analysis confirmed that our model focused on clinically relevant cues aligned with KTAS criteria.

Conclusion: BERT-based LLMs trained on real-world ED conversations significantly outperform general-purpose models like ChatGPT in triage accuracy. This approach demonstrates the potential for enhancing clinical decision support with interpretable and efficient AI.

Objectives: To develop and evaluate a human-LLM (Large Language Model) collaborative approach for systematic ontology updating, demonstrated with the Dietary Lifestyle Ontology (DILON).

Materials and methods: One hundred dietary questionnaire items from English and Korean sources were semantically annotated by 4 state-of-the-art language models, which generated candidate concepts for inclusion into DILON. Outputs were refined through cross-model reconciliation, followed by expert review. The model curated the concept within DILON and experts reviewed and refined the outputs in Protégé to ensure accuracy and consistency.

Results: Claude Sonnet 4 effectively supported local tasks, including harvesting new concepts, detecting redundancies, and refining hierarchical segments. Global optimization of ontology, however, required systematic examination by human experts.

Discussion: These findings highlight the complementary strengths of LLMs and humans: LLMs accelerate repetitive and local updates, whereas humans maintain overall structural integrity.

Conclusion: Human-LLM collaboration improves efficiency, scalability, and sustainability in ontology engineering, supporting the maintenance of complex biomedical ontologies.

Background and significance: Ambient listening tools powered by generative artificial intelligence (GenAI) offer real-time, scribe-like support that reduce documentation burden and may help alleviate burnout. This study assesses physician-perceived benefits and challenges of ambient AI implementation through surveys and evaluates its effectiveness in clinical workflows using automatically recorded electronic health record (EHR) time-efficiency metrics.

Method and materials: A quality improvement pilot has been underway at UCI Health since December 2023. Epic EHR Signal metrics were analyzed to assess changes in note length, documentation time, and same-day encounter closure rates. Matched pre- and post-implementation surveys evaluated physician-perceived changes in documentation burden, clinical efficiency, and care quality. We also examined open-ended survey responses using thematic analysis to supplement quantitative findings.

Results: Analysis on EHR usage data from 167 physicians showed significant reductions in note-writing time, despite an increase in note length. Survey responses (n = 65) also indicated statistically significant improvements across multiple domains. Physicians reported reduced cognitive demand (P = .031) and documentation effort (P = .014), alongside perceptions of enhanced clinical efficiency, patient-centered care, and EHR system usability. Thematic analysis confirmed these quantitative findings and identified opportunities for improvement, including specialty-specific customization and expanded AI functionality.

Discussion: Ambient AI tools demonstrated improved documentation efficiency, perceived care quality, and reduced cognitive workload. These benefits suggest potential to alleviate key burdens in clinical documentation.

Conclusion: Future development should prioritize customization for specialty-specific and individual physician needs, ensure the reliability and accuracy of AI-generated content, and integrate ethical and legal considerations to facilitate safe and scalable implementation in patient-centered care contexts.

Objective: Artificial intelligence (AI) has impacted healthcare at urban and academic medical centers in the US. There are concerns, however, that the promise of AI may not be realized in rural communities. This scoping review aims to determine the extent of AI research in the rural US.

Materials and methods: We conducted a scoping review following the PRISMA guidelines. We included peer-reviewed, original research studies indexed in PubMed, Embase, and WebOfScience after January 1, 2010 and through April 29, 2025. Studies were required to discuss the development, implementation, or evaluation of AI tools in rural US healthcare, including frameworks that help facilitate AI development (eg, data warehouses).

Results: Our search strategy found 26 studies meeting inclusion criteria after full text screening with 14 papers discussing predictive AI models and 12 papers discussing data or research infrastructure. AI models most often targeted resource allocation and distribution. Few studies explored model deployment and impact. Half noted the lack of data and analytic resources as a limitation. None of the studies discussed examples of generative AI being trained, evaluated, or deployed in a rural setting.

Discussion: Practical limitations may be influencing and limiting the types of AI models evaluated in the rural US. Validation of tools in the rural US was underwhelming.

Conclusion: With few studies moving beyond AI model design and development stages, there are clear gaps in our understanding of how to reliably validate, deploy, and sustain AI models in rural settings to advance health in all communities.