JAMIA Open最新文献

Background: Recent advancements of Artificial Intelligence (AI) are rapidly transforming clinical research. While this technology offers exciting opportunities, it amplifies existing concerns regarding the need for transparent methodology that fosters patient engagement, and introduces new challenges. PCORI's Improving Methods portfolio has invested in methodological research to enhance rigor and transparency via patient-centered approaches in AI.

Objective: This commentary outlines PCORI's approach to funding and promoting a portfolio of methodological research that aims to improve the conduct of patient-centered comparative clinical effectiveness research (CER), with a focus on AI methods. The paper highlights a growing portfolio of over 40 AI related projects, including a recent cohort leveraging large language models to augment research processes in CER.

Discussion: PCORI's current portfolio of methods projects in AI illustrate timely opportunities for the clinical research informatics community to develop and assess AI applications that will further advance a robust, interoperable and ethical infrastructure for patient-centered CER. PCORI's requirement for ongoing, meaningful engagement of patients throughout the research lifecycle provides a blueprint for patient-centered AI by developing and applying models and methods designed to create value for patients and other healthcare partners.

Objective: Systemic lupus erythematosus (SLE) is a chronic autoimmune disease that disproportionately affects women and racial/ethnic minority groups. Predicting disease flares is essential for improving patient outcomes, yet few studies integrate both clinical and social determinants of health (SDoH). We therefore developed FLAME (FLAre Machine learning prediction of SLE), a machine learning pipeline that uses electronic health records (EHRs) and contextual-level SDoH to predict 3-month flare risk, emphasizing explainability and fairness.

Materials and methods: We conducted a retrospective cohort study of 28 433 patients with SLE from the University of Florida Health (2011-2022), linked to 675 contextual-level SDoH variables. We used XGBoost and logistic regression models to predict 3-month flare risk, evaluating model performance using the area under the receiver operating characteristic (AUROC). We applied SHapley Additive exPlanations (SHAP) values and causal structure learning to identify key predictors. Fairness was assessed using the equality of opportunity metric, measured by the false-negative rate across racial/ethnic groups.

Results: The FLAME model, incorporating clinical and contextual-level SDoH, achieved an AUROC of 0.66. The clinical-only model performed slightly better (AUROC of 0.67), while the SDoH-only model had lower performance (AUROC of 0.54). SHAP analysis identified headache, organic brain syndrome, and pyuria as key predictors. Causal learning revealed interactions between clinical factors and contextual-level SDoH. Fairness assessments showed no significant biases across groups.

Discussion: FLAME offers a fair and interpretable approach to predicting SLE flares, providing meaningful insights that may guide future clinical interventions.

Conclusions: FLAME shows promise as an EHR-based tool to support personalized, equitable, and holistic SLE care.

Objective: Our objective was to determine how social vulnerabilities, broadband access, and rurality relate to telemedicine use across the United States through large-scale analysis of real-world telemedicine data.

Materials and methods: We conducted a retrospective, observational study of dyadic U.S. telemedicine sessions that occurred January 1, 2022 to December 31, 2022, linked to the 2020 Centers for Disease Control and Prevention Social Vulnerability Index (SVI) and the National Center for Health Statistics Urban-Rural Classification Scheme for Counties. We examined county-level telemedicine use rates (sessions per 1000 population) in relation to SVI indexes, broadband internet access, and rurality classifications using polynomial regression and data visualization.

Results: We found a negative, nonlinear association between overall social and socioeconomic status vulnerabilities and telemedicine use. Telemedicine rates in urban counties exceeded that of rural counties. There was more variability in telemedicine use for the urban counties according to social vulnerability and broadband access.

Discussion: Rurality and broadband access demonstrated a greater effect on telemedicine use than social vulnerability, and the relationship between social vulnerability, broadband access, and telemedicine use differed for rural versus urban areas.

Conclusion: This observational study of nearly 8 million U.S. telemedicine sessions showed that rurality and broadband access are key drivers of telemedicine use and may be more important than many social vulnerabilities in determining community-level telemedicine use. We also found nuanced differences in the relationship between social vulnerability and telemedicine use between rural and urban counties, and at different levels of broadband access.

Objective: Digital dashboards are used to monitor patients and improve inpatient outcomes in hospital settings. A systematic review assessed the impact of dashboards across five outcomes of hospital mortality, hospital length of stay (LOS), economic impacts, harms, and patient and carer satisfaction.

Materials and methods: Nine databases were searched from inception to May 2024. Studies were included if they reported primary quantitative research on dashboard interventions in hospital settings, were in English, and measured effectiveness for patients, caregivers, healthcare professionals or services. Data synthesis was performed via narrative review. Risk of bias was measured using Cochrane ROBINS-I and RoB 2.

Results: We identified 5755 articles, and 70 met inclusion criteria. Of 20 findings reporting mortality (16 studies), five reported a decrease, whilst the majority (n = 15) found no significant change. LOS was reported across 43 findings (31 studies), with 28 reporting a reduction, an increase in five, and ten reporting no change. Of 21 findings (from 16 studies) reporting on harms, increases were observed in six, decreases in four, and no change in 11. Economic impacts were reported in 34 findings (31 studies), with the majority demonstrating reduced costs (n = 29), an increase in one, and no change in four. Eight findings (eight studies) reported on patient and carer satisfaction with care, with the majority (n = 6) demonstrating increased satisfaction, and two reporting no change.

Discussion: Hospital dashboards do appear associated with either no change or a reduction in mortality, reduced costs, reduced LOS, and improved patient and caregiver satisfaction with care. Association with harms was equivocal.

Conclusion: While there is evidence of potential benefits, actual impacts of hospital digital dashboard will likely be dependent on multiple local factors such as workflow integration.

Objective: To develop and evaluate machine learning (ML) models for predicting length of stay (LOS) in elective spine surgery, with a focus on the benefits of temporal modeling and model interpretability.

Materials and methods: We compared traditional ML models (eg, Linear Regression, Random Forest, Support Vector Machine [SVM], and XGBoost) with our developed model, SurgeryLSTM, a masked bidirectional long short-term memory (BiLSTM) with an attention, using structured perioperative electronic health records (EHR) data. Performance was evaluated using the coefficient of determination (R ²), and key predictors were identified using explainable AI.

Results: SurgeryLSTM achieved the highest predictive accuracy (R ² = 0.86), outperforming XGBoost (R ² = 0.85) and baseline models. The attention mechanism improved interpretability by dynamically identifying influential temporal segments within preoperative clinical sequences, allowing clinicians to trace which events or features most contributed to each LOS prediction. Key predictors of LOS included bone disorder, chronic kidney disease, and lumbar fusion identified as the most impactful predictors of LOS.

Discussion: Temporal modeling with attention mechanisms significantly improves LOS prediction by capturing the sequential nature of patient data. Unlike static models, SurgeryLSTM provides both higher accuracy and greater interpretability, which are critical for clinical adoption. These results highlight the potential of integrating attention-based temporal models into hospital planning workflows.

Conclusion: SurgeryLSTM presents an effective and interpretable AI solution for LOS prediction in elective spine surgery. Our findings support the integration of temporal, explainable ML approaches into clinical decision support systems to enhance discharge readiness and individualized patient care.

Objectives: To develop and test a novel machine learning approach for monitoring impact of computerized clinical decision support (CDS) tools on clinicians' electronic health record (EHR) activities.

Materials and methods: Our CDS monitoring approach leverages topic modeling, a latent-variable statistical machine learning method, to infer health providers' EHR activities from EHR audit logs. We applied this approach to monitor the impact of a tobacco cessation support CDS tool newly implemented in 5 cancer clinics (2018-2021). We trained the topic model on EHR audit log data from 3445 encounters (pre-CDS-implementation: 1734, post-CDS-implementation: 1711) for patients with active smoking status. The number of topics was automatically determined based on within-topic coherence and across-topic divergence, and the identified topics were assigned clinically relevant EHR activity labels by 4 domain experts.

Results: The topic model identified 2 distinct activities focusing on CDS (act on CDS, bypass/postpone CDS), 2 activities related to CDS (review patient records and address alerts, use note templates and acknowledge the completion of CDS), 6 related to accessing (access patient station) and reviewing patient data (external records, synopsis data, snapshot of patient data, problem list/diagnosis/notes, treatment plan), and 4 related to modifying EHR (modify diagnosis/problem lists, document visit with record review, perform administrative activities for visit and billing, and document follow-up care plan). Comparing matched 1-hour after-check-in windows post-implementation (n = 841) versus pre-implementation (n = 841) of CDS, the mean prevalence (expressed as proportions out of 1.0) of providers' EHR-use activity increased on CDS-focused activities (0.073, 95% CI, 0.066-0.079) and CDS-related activities (0.098, 95% CI, 0.089-0.106) and decreased on modifying EHR (-0.113, 95% CI, -0.124 to -0.102) and reviewing patient data (-0.058, 95% CI, -0.072 to -0.044).

Discussion: Our topic model-based CDS monitoring approach can identify shifts in prevalence of EHR-use activities pre-implementation versus post-implementation. This approach can be applied to detect unintended changes in EHR activities on a large population scale following CDS implementation, providing valuable insights to guide focused qualitative investigations for CDS improvement or de-implementation.

Conclusion: Our approach offers a scalable, data-driven framework for evaluating the real-world impact of EHR-embedded CDS tools. Built on a generic machine learning framework, this approach could be adapted to explore impact of other healthcare quality improvement strategies using EHR-integrated CDS interventions.

Background: While most health-care providers now use electronic health records (EHRs) to document clinical care, many still treat them as digital versions of paper records. As a result, documentation often remains unstructured, with free-text entries in progress notes. This limits the potential for secondary use and analysis, as machine-learning and data analysis algorithms are more effective with structured data.

Objective: This study aims to use advanced artificial intelligence (AI) and natural language processing (NLP) techniques to improve diagnostic information extraction from clinical notes in a periodontal use case. By automating this process, the study seeks to reduce missing data in dental records and minimize the need for extensive manual annotation, a long-standing barrier to widespread NLP deployment in dental data extraction.

Materials and methods: This research utilizes large language models (LLMs), specifically Generative Pretrained Transformer 4, to generate synthetic medical notes for fine-tuning a RoBERTa model. This model was trained to better interpret and process dental language, with particular attention to periodontal diagnoses. Model performance was evaluated by manually reviewing 360 clinical notes randomly selected from each of the participating site's dataset.

Results: The results demonstrated high accuracy of periodontal diagnosis data extraction, with the sites 1 and 2 achieving a weighted average score of 0.97-0.98. This performance held for all dimensions of periodontal diagnosis in terms of stage, grade, and extent.

Discussion: Synthetic data effectively reduced manual annotation needs while preserving model quality. Generalizability across institutions suggests viability for broader adoption, though future work is needed to improve contextual understanding.

Conclusion: The study highlights the potential transformative impact of AI and NLP on health-care research. Most clinical documentation (40%-80%) is free text. Scaling our method could enhance clinical data reuse.

Objectives: Measles continues to pose a serious threat to global public health, fueled by declining vaccination rates, international travel, and persistent immunization gaps. Early outbreak detection and response remain hampered by fragmented surveillance systems, which often lack interoperability and limit data accessibility.

Materials and methods: To address the major limitations of current measles surveillance systems-including data fragmentation and lack of standardization-we developed Measles Tracker, an integrated near-real-time data hub that centralizes and harmonizes measles surveillance data in the United States using publicly available sources. The system aggregates data from multiple layers, including: (1) official reports from public health agencies, (2) epidemiological surveillance bulletins, and (3) outbreak reports, mainly captured through news websites or via news aggregators. The platform architecture implements (1) geospatial normalization of key epidemiological variables (case counts, vaccination coverage, age-stratified incidence) and (2) dynamic visualization interfaces to support coordination of evidence-based response.

Results: Measles Tracker enhances situational awareness by integrating disparate data streams in near real-time, enabling rapid geospatial detection of outbreak clusters, mapping vaccination gaps, and supporting dynamic risk stratification of vulnerable populations. It is intended exclusively as a complementary tool to official public health systems, providing educational and situational awareness without interfering with contact tracing, vaccination, or outbreak control activities.

Conclusions: As a centralized, scalable tool, Measles Tracker advances measles surveillance by leveraging digital epidemiology principles. Future iterations will incorporate additional data streams (eg, climate variables, genomic surveillance) and advanced analytics (eg, machine learning for risk prediction, network models for transmission dynamics) to further optimize outbreak preparedness and resource allocation. This framework underscores the transformative potential of integrated data systems in global measles elimination efforts.

Objective: This study compares the performance of machine learning (ML) models and human experts in mapping unstructured nursing notes to the standardized Nursing Interventions Classification (NIC) system. The aim is to advance automated nursing documentation classification, facilitating cross-facility benchmarking of patient care and organizational outcomes.

Materials and methods: We developed and compared 4 ML models: TF-IDF text-based vectorization, UMLS semantic mapping, fine-tuned GPT-4o mini, and Bio-Clinical BERT. These models were evaluated against classifications provided by 2 expert nurses using a dataset of de-identified home healthcare nursing notes obtained from a Florida, USA-based medical clearinghouse. Model performance was assessed using agreement statistics, precision, recall, F1 scores, and Cohen's Kappa.

Results: Human raters achieved the highest agreement with consensus labels, scoring 0.75 and 0.62, with corresponding F1 scores of 0.61 and 0.45, respectively. In comparison, ML models showed lower performance, with GPT achieving the best among them (agreement: 0.50, F1 score: 0.31). A distribution analysis of NIC categories revealed that ML models performed well in prevalent and clearly defined categories, such as drug management, but struggled with minority classes and context-dependent interventions, like information management.

Discussion: Current ML approaches show promise in supporting clinical classification tasks, but the performance gap in handling complex, context-dependent interventions highlights the need for improved methods that can better capture the nuanced nature of clinical documentation. Future research should focus on developing methods to process clinical terminology and context-specific documentation with greater precision and adaptability.

Conclusion: Current ML models can aid-but not fully replace-human judgment in classifying nuanced nursing interventions.

Objective: The International Classification of Health Interventions (ICHI), currently being developed, seeks to span all sectors of the health system. Our objective was to determine the coverage of the ICHI for hearing interventions commonly delivered to adults with sensorineural hearing loss (SNHL).

Material and methods: A 3-phase content mapping method was used, which included (1) identification of source terms with an expert panel in audiology rehabilitation; (2) 3 coders independently applied the classification to the source terms; and (3) the coders reached a consensus for each intervention and identified reasons for initial discrepancies with options not linked to a specific code were identified.

Results: Nineteen different ICHI Target categories were identified, with 23 different ICHI Action categories and 82% of the means being "Other and unspecified." There was consensus in codes for 54.3% of source terms, with no ICHI code found for 8.5% of source terms. The greatest number of discrepancies arose from the action, followed by the target. Coding discrepancies occurred as a result of misunderstanding of source terms, the clinical use thereof, and difficulty determining the type of Target.

Discussion: Despite its broad scope, ICHI's current framework has gaps in its coverage of audiological interventions, particularly those related to sensorineural hearing loss. Addressing these gaps is crucial for improving global data standardization and facilitating the development of more targeted hearing health policies.

Conclusion: This study makes an important contribution to the further development and refinement of the classification, specifically in the context of hearing healthcare.