Journal of the American Medical Informatics Association最新文献

Objectives: To develop and evaluate a knowledge graph-augmented large language model (LLM) framework that synthesizes epidemiological evidence to infer life-course exposure-outcome pathways, using gestational diabetes mellitus (GDM) and dementia as a case study.

Materials and methods: We constructed a causal knowledge graph by extracting empirical epidemiological associations from scientific literature, excluding hypothetical assertions. The graph was integrated with GPT-4 through four graph retrieval-augmented generation (GRAG) strategies to infer bridging variables between early-life exposure (GDM) and later-life outcome (dementia). Semantic triples served as structured inputs to support LLM reasoning. Each GRAG strategy was evaluated by human clinical experts and three LLM-based reviewers (GPT-4o, Llama 3-70B, and Gemini Advanced), assessing scientific reliability, novelty, and clinical relevance.

Results: The GRAG strategy using a minimal set of abstracts specifically related to GDM-dementia bridging variables performed comparably to the strategy using broader sub-community abstracts, and both significantly outperformed approaches using the full GDM- or dementia-related corpus or baseline GPT-4 without external augmentation. The knowledge graph-augmented LLM identified 108 maternal candidate mediators, including validated risk factors such as chronic kidney disease and physical inactivity. The structured approach improved accuracy and reduced confabulation compared to standard LLM outputs.

Discussion: Our findings suggest that augmenting LLMs with epidemiological knowledge graphs enables effective reasoning over fragmented literature and supports the reconstruction of progressive risk pathways. Expert assessments revealed that LLMs may overestimate clinical relevance, highlighting the need for human-AI collaboration in interpretation and application.

Conclusion: Integrating semantic epidemiological knowledge with LLMs via GRAG strategies provides a promising framework for life-course epidemiology, enabling early detection of modifiable risk factors and guiding variable selection in cohort study design.

Objectives: Documentation of gender identity (GI) and anatomy data in the electronic health record (EHR) is a proposed standard of care for transgender populations. However, there is limited research on implementation of proposed best practices, particularly anatomy data collection. This study aims to characterize factors that influence patient preferences and comfort around the collection and documentation of GI and anatomy in EHRs.

Materials and methods: From November 2023 to January 2024, 17 one-on-one, semi-structured virtual interviews were conducted with transgender adults residing in the Metropolitan New York area. Transcriptions were analyzed using inductive thematic analysis.

Results: Themes clustered around comfort and preferences for data collection processes and outcomes. Factors that influenced preferences and comfort around anatomy data were distinct from those impacting GI documentation preferences and comfort. The tension between the categories of GI and sex assigned at birth impacted anatomy data documentation preferences. Clinical context emerged as a consistent factor that impacts both preferences and comfort of GI and anatomy data documentation.

Discussion and conclusion: GI data collection efforts in clinical settings must consider the implication of anatomy data collection when determining data collection best practice methodologies. Anticipated and experienced stigma remain significant hurdles to patient comfort and willingness to collect GI and anatomy data, and their impact on actual data collection should be further elucidated among diverse gender identities. Clinical data collection methods, tools, and education warrant ongoing research investment to further elucidate best practices.

Objective: To assess patient attitudes towards ambient artificial intelligence (AI) scribes, including comfort, trust, perceived impact on provider interactions, and willingness for future use, and to examine how sociodemographic, health factors, digital literacy, and privacy concerns shape attitudes.

Materials and methods: We analyzed cross-sectional data from an online survey of 12 153 adults (52.4% female; 23.1% aged ≥ 65; 41.2% with chronic conditions) in Canada conducted between February 6 and March 10, 2025. Survey-adjusted ordinal and binary logistic regression models assessed predictors, reporting adjusted odds ratios (aORs), 95% confidence intervals (CIs), and P-values.

Results: Most respondents (61.8%) were reluctant to future AI scribe use despite mixed attitudes: 39.3% reported some/very high comfort, 57.4% trusted documentation with human oversight, and 49.5% anticipated positive effects on patient-provider interactions. Awareness of AI scribe use was low (28.3%). Males showed higher odds of favorable comfort (aOR = 1.13, 95% CI, 1.05-1.22, P = .001), trust (aOR = 1.21, 95% CI, 1.10-1.32, P < .001), and future use (aOR = 1.38, 95% CI, 1.27-1.51, P < .001). Chronic conditions showed higher odds of future use (aOR = 1.19, 95% CI, 1.08-1.32, P < .001), whereas poorer general health was associated with lower odds across all outcomes. Fewer emergency room/urgent care visits, lower education, and income levels were associated with less favorable attitudes across outcomes. Higher digital health literacy (aOR = 1.03-1.04, all P < .001) and AI knowledge (aOR = 1.28-1.37, all P < .001) showed associations with higher odds across outcomes; privacy concerns were linked to lower odds (eg, future use: aOR = 0.65, 95% CI, 0.63-0.68, P < .001).

Discussion: Findings reveal a paradox-patients expressed conditional trust and comfort yet remained reluctant to adopt AI scribes, with privacy concerns and low awareness as key barriers.

Conclusion: Targeted interventions addressing digital literacy, privacy safeguards, and clinician-patient communication about AI scribes are needed before widespread adoption.

Objectives: To qualitatively characterize barriers and facilitators to implementing and using an ambient scribe across a large academic medical center, as well as how ambient transcription reshapes clinicians' perceptions of their work.

Materials and methods: We conducted semistructured interviews with clinicians who participated in an ambient scribe pilot (n = 8) and the initial enterprise rollout (n = 16). We sought heterogeneity by specialty, note volume, burnout, and prior time-in-notes. Interviews (26-60 min) were recorded, transcribed, and analyzed thematically using a naturalistic, ethnographic approach informed by broad implementation considerations, and an analytic lens treating note sections as documentation "genres."

Results: Clinicians described feeling more present with patients and greater satisfaction during visits. Fictions included overlong or underspecified sections (eg, History of Present Illness vs Assessment & Plan), unfamiliar formatting, and a perceived loss of "voice." Participants discussed how they used documentation to personalize practice, demonstrate expertise, manage impressions with colleagues and supervisors, and communicate sensitive findings-activities not fully captured by efficiency metrics. Inpatient and procedure-heavy contexts reported limited benefit where documentation was already highly standardized.

Discussion: Early ambient scribe implementation produced recognizable benefits, but introduced new work to reconcile AI-drafted text with local documentation genres and audience-specific communication. Tailored prompts, onboarding, and peer support may reduce the need to revise artificial intelligence (AI)-generated text.

Conclusion: Ambient scribe adoption can enhance patient interactions and perceived efficiency while reshaping how clinicians express voice and expertise in notes. Implementation strategies attentive to documentation genre and audience may help align ambient scribe outputs with clinical communication needs.

Objectives: Real-world evidence (RWE) increasingly informs clinical decisions, yet manual adjustment for confounding limits scalability. Data-adaptive (DA) algorithms for high-dimensional proxy adjustment show promise but have not been systematically compared to investigator-specified (IS) approaches across diverse treatment scenarios. We evaluated whether DA strategies perform comparably to manually curated IS models using claims-based emulations of 15 randomized trials from the RCT-DUPLICATE initiative.

Materials and methods: We identified new-user cohorts for 15 trial emulations in Optum's de-identified Clinformatics Data Mart Database (2004-2023). Treatment effects were estimated using 3 adjustment strategies: (1) IS models with manually tailored covariates; (2) full-DA strategies using empirical features from semiautomated pipelines; and (3) hybrid-DA models incorporating both empirical and investigator-defined covariates. Agreement with RCT benchmarks was assessed via binary metrics and difference-in-differences.

Results: Outcome-adaptive LASSO achieved better RWE-RCT agreement than IS adjustment in 73% of full-DA and 87% of hybrid-DA emulations. Other DA methods considering feature associations with both treatment and outcome performed similarly well, while models tuned solely for treatment prediction performed poorly. Performance of IS vs DA strategies differed across emulated trials.

Discussion: Top DA algorithms matched manual IS models on average, but impact varied by emulation. Case studies illustrate the continued importance of subject-matter knowledge, particularly for complex treatment strategies.

Conclusion: Data-adaptive algorithms show promise for scalable confounding adjustment in large-scale evidence systems and as augmentation tools for investigator-specified designs. Hybrid strategies combining algorithmic methods with investigator expertise offer the most reliable approach for individual causal questions.

Introduction: Automation of clinical orders in electronic health records (EHRs) has the potential to reduce clinician burden and enhance patient safety. However, determining which orders are appropriate for automation requires a structured framework to ensure clinical validity, transparency, and safety.

Objective: To develop and validate a framework of desiderata for assessing the appropriateness of automating clinical orders in EHRs and to demonstrate its operational value in a live health system dataset.

Materials and methods: The study comprised 4 phases to move from concept generation to real-world demonstration. First, we conducted focus group analyses using ground theory to identify themes and developed desiderata informed by these themes and existing literature. We validated the desiderata by surveying clinicians at a single institution, presenting 10 use cases to and assessing perceived appropriateness, cognitive support, and patient safety using a 4-point Likert scale. Survey results were compared to a priori appropriateness designations using t-tests. To evaluate operational impact, we analyzed one year of order-based alerts and orders (1.4 million firings alert and 44.1 million orders, respectively) using filtering rules and association rule mining to identify candidate orders for automation and their impact.

Results: We identified 8 desiderata for automated order appropriateness: logical consistency, data provenance, order transparency, context permanence, monitoring plans, trigger consistency, care team empowerment, and system accountability. Use cases deemed appropriate based on these criteria received significantly higher scores for appropriateness (3.13 ± 0.84 vs 2.30 ± 0.99), cognitive support (3.08 ± 0.82 vs 2.25 ± 0.94), and patient safety (3.08 ± 0.86 vs 2.21 ± 0.98) (all P < .001) compared to those considered inappropriate. Operational analysis revealed an alert firing 19 109 times annually, with a 96% signed order rate, where automation could save an estimated 26.5 provider hours per year. Additionally, an association rule with 16 628 occurrences (68.4% confidence) suggested automation could save 15.8 hours annually and yield 8000 additional appropriate orders.

Discussion: The desiderata align with clinician perceptions and provide a structured approach for evaluating automated orders. Our findings highlight the potential for automation of certain clinical orders to improve cognitive support while maintaining patient safety.

Conclusion: Healthcare systems should use these desiderata, coupled with data mining techniques, to systematically identify and govern appropriate automated orders. Further research is needed to validate operational scalability.

Objectives: Externalizing behaviors in children, such as aggression, hyperactivity, and defiance, are influenced by complex interplays between genetic predispositions and environmental factors, particularly parental behaviors. Unraveling these intricate causal relationships can benefit from the use of robust data-driven methods.

Materials and methods: We developed "Hillclimb-Causal Inference," a causal discovery approach that integrates the Hill Climb Search algorithm with a customized Linear Gaussian Bayesian Information Criterion (BIC). This method was applied to data from the Adolescent Brain Cognitive Development (ABCD) Study, which included parental behavior assessments, children's genotypes, and externalizing behavior measures. We performed dimensionality reduction to address multicollinearity among parental behaviors and assessed children's genetic risk for externalizing disorders using polygenic risk scores (PRS), which were computed based on GWAS summary statistics from independent cohorts. Once the causal pathways were identified, we employed structural equation modeling (SEM) to quantify the relationships within the model.

Results: We identified prominent causal pathways linking parental behaviors to children's externalizing outcomes. Parental alcohol misuse and broader behavioral issues exhibited notably stronger direct effects (0.33 and 0.20, respectively) compared to children's PRS (0.07). Moreover, when considering both direct and indirect paths, parental substance misuse (alcohol, drugs, and tobacco) collectively resulted in a total effect exceeding 1.1 on externalizing behaviors. Bootstrap and sensitivity analyses further validated the robustness of these findings.

Discussion and conclusion: Parental behaviors exert larger effects on children's externalizing outcomes than genetic risk, suggesting potential targets for prevention and intervention. The Hillclimb-Causal framework provides a general, data-driven way to map causal pathways in developmental psychiatry and related domains.

Objectives: Longitudinal data in electronic health records (EHRs) represent an individual's clinical history through a sequence of codified concepts, including diagnoses, procedures, medications, and laboratory tests. Generative pretrained transformers (GPT) can leverage this data to predict future events. While fine-tuning of these models can enhance task-specific performance, it becomes costly when applied to many clinical prediction tasks. In contrast, a pretrained foundation model can be used in zero-shot forecasting setting, offering a scalable alternative to fine-tuning separate models for each outcome.

Materials and methods: This study presents the first comprehensive analysis of zero-shot forecasting with GPT-based foundational models in EHRs, introducing a novel pipeline that formulates medical concept prediction as a generative modeling task. Unlike supervised approaches requiring extensive labeled data, our method enables the model to forecast the next medical event purely from a pretraining knowledge. We evaluate performance across multiple time horizons and clinical categories, demonstrating model's ability to capture latent temporal dependencies and complex patient trajectories without task supervision.

Results: The model's performance in predicting the next medical concept was evaluated using precision and recall metrics, achieving an average top-1 precision of 0.614 and recall of 0.524. For 12 major diagnostic conditions, the model demonstrated strong zero-shot performance, achieving high true positive rates while maintaining low false positives.

Discussion: We demonstrate the power of a foundational EHR GPT model in capturing diverse phenotypes and enabling robust, zero-shot forecasting of clinical outcomes. This capability highlights both its versatility across conditions like liver cancer and SLE, and its limitations in more ambiguous settings such as depression, while also revealing meaningful latent clinical structure.

Conclusion: This capability enhances the versatility of predictive healthcare models and reduces the need for task-specific training, enabling more scalable applications in clinical settings.

Background and significance: Acute otitis media (AOM) is a leading cause of pediatric antibiotic overuse. Safety Net Antibiotic Prescriptions (SNAPs) are recommended for antibiotic stewardship but are difficult to identify due to lack of structured documentation.

Objective: This study validates the accuracy of Versa, a GPT-4o based HIPAA-compliant large language model (LLM), to classify AOM treatment plans from physician notes.

Methods: A retrospective cross-sectional study analyzed pediatric AOM encounters. Multiple prompting strategies were used to classify treatment plans and validated against a representative sample of manual reviews by 2 pediatricians. A locally fine-tuned model, Clinical-Longformer was also trained and tested against Versa and human review.

Results: In total, 5707 encounters were included; 374 reviewed manually. Zero-shot accuracy was 97.8%; few-shot accuracy was 85%. Clinical-Longformer achieved 93.3% accuracy.

Conclusion: Versa effectively identifies AOM treatment plans, providing a cost-efficient quality improvement tracking tool for prescription practice patterns in pediatric antibiotic stewardship efforts.