Journal of the American Medical Informatics Association最新文献

Objective: This study evaluates the impact of self-management and support of m-health applications on medication adherence (MA) and the corresponding long-term medical expenditures among patients with Type 2 Diabetes (T2D), using an analytic framework generalizable to other chronic conditions.

Materials and methods: A systematic review and meta-analysis of randomized controlled trials were conducted to estimate the synthesized effect of m-health interventions on MA. These results were integrated into a Markov state-transition model to simulate patient transitions among three adherence levels over a 10-year horizon. Medical expenditure data by adherence level were derived from the Medical Expenditure Panel Survey (MEPS). Monte Carlo simulation was applied to assess uncertainty and estimate individual- and population-level cost outcomes under baseline and intervention scenarios.

Results: The meta-analysis showed a significant positive effect of m-health on MA (standardized mean difference = 0.21, 95% CI: 0.14-0.28). Patients in the intervention scenario experienced an average cost reduction of $4400 over 10 years. At the population level, a cohort of 10 000 patients using m-health tools would yield projected direct medical cost savings of $44 million.

Discussion: This study demonstrates the potential of m-health interventions to improve patient behavior and generate substantial long-term cost savings. By linking behavioral health data to downstream cost outcomes, the study adds to the growing evidence base for informatics-driven population health strategies.

Conclusion: Our study underscores the importance of integrating digital support tools into chronic disease care and informs policy decisions aimed at integrating health informatics innovations.

Objective: To develop the first comprehensive, standardized annotated corpus of Chinese online health information (OHI) using the full 16-item DISCERN instrument and to establish a reliable annotation process that supports automated quality assessment.

Materials and methods: We assembled 510 web-sourced articles on breast cancer, arthritis, and depression. All the articles were independently annotated by three trained raters using the DISCERN scale. Annotation followed a four-step workflow: data collection and preprocessing, rater training, iterative annotation, and quality control. Raters calibrated through consensus sessions and calibration articles. The Dawid-Skene model aggregated individual annotations into final consensus scores. Original five-point ratings were retained and binarized (scores 1-3 as low quality, 4-5 as high quality) to enable both fine-grained and coarse evaluation for machine learning.

Results: Initial annotation of a 60-article pilot produced low agreement (mean Krippendorff's α ≈ 0.022) due to subjective variability. Successive calibration exercises improved agreement markedly, culminating in a corpus-wide Krippendorff's α of 0.834. Consensus scores correlated strongly with individual rater scores, confirming annotation robustness. The dual-scale design yielded a relatively balanced distribution of labels across topics, with roughly equal representation of low- and high-quality articles, and preserved granularity for detailed DISCERN analysis.

Discussion: Our iterative calibration approach and consensus modeling effectively addressed the subjective ambiguity inherent in quality assessment. The binary and five-class labeling strategies facilitate flexible downstream applications, allowing automated systems to perform both broad filtering and nuanced quality differentiation. The high inter-rater reliability demonstrates that rigorous training and consensus methods can overcome domain-specific annotation challenges.

Conclusion: The resulting Chinese OHI corpus, annotated via a standardized DISCERN framework and refined through iterative calibration, provides a robust benchmark for training and evaluating machine learning models. This resource lays the foundation for scalable, reliable automated quality assessment of OHI in Chinese public health settings.

Objectives: Patients value access to their medical reports on patient portals, but the terminology in those reports can cause confusion and anxiety. Can the artificial intelligence (AI) simplification of radiology reports into plain language improve patient comprehension?

Materials and methods: Twenty original radiology reports (breast imaging, chest X-ray) were simplified into plain language using ChatGPT-4 using a customized prompt for each report type. For each report, clinicians created a gold standard of key findings and appropriate follow-up. In August 2024, a national sample of 2000 US adults reviewed 2 randomly assigned reports, 1 original and 1 AI-generated plain language. Participants answered questions focused on comprehension of key findings, follow-up, confidence, anxiety, and preferences. Comprehension and follow-up were compared to the gold standard. We compared patient accuracy for original vs AI-generated plain language reports.

Results: Participants (mean age 48 years) were 62.3% female. Compared to original reports, participants shown AI-generated plain language reports had higher accuracy in comprehension (68.0% vs 58.0%; marginal difference 10.8% [95% CI, 7.8%-13.8%]) and follow-up (64.5% vs 58.4%; marginal difference 6.8% [95% CI, 4.1%-9.4%]). Improvements were larger among participants aged >44 years and with less than college education. With plain language reports, participants reported higher confidence in their answers and lower anxiety. Despite these improvements, 60.0% of participants preferred the original report over the plain language version.

Discussion: Integrating AI simplification into patient portals may be helpful, but trust concerns remain.

Conclusion: AI simplification improved patient comprehension and confidence. Further research is needed to address patient resistance to AI simplification.

Background: Computational phenotyping from electronic health records (EHRs) is essential for clinical research, decision support, and quality/population health assessment, but the proliferation of algorithms for the same conditions makes it difficult to identify which algorithm is most appropriate for reuse.

Objective: To develop a framework for assessing phenotyping algorithm fitness for purpose and reuse.

Fitness for purpose: Phenotyping algorithms are fit for purpose when they identify the intended population with performance characteristics appropriate for the intended application.

Fitness for reuse: Phenotyping algorithms are fit for reuse when the algorithm is implementable and generalizable-that is, it identifies the same intended population with similar performance characteristics when applied to a new setting.

Conclusions: The PhenoFit framework provides a structured approach to evaluate and adapt phenotyping algorithms for new contexts increasing efficiency and consistency of identifying patient populations from EHRs.

Objective: To introduce a novel, standardised approach to evaluating AI prediction models in balancing effectiveness, efficiency and utility, using a sepsis prediction model case study.

Materials and methods: Retrospective patient data from electronic medical records of 7 public hospitals was used to retrain and evaluate a machine learning sepsis prediction model. Four conventional metrics-area under the receiver operating curve (AUROC), sensitivity, positive predictive value, and specificity-were compared with a novel graphical display integrating metrics of predictive accuracy (effectiveness), alert burden (efficiency) and lead time of alerts relative to clinical events (utility) for different alert thresholds.

Results: The dataset comprised 977,506 inpatient admissions. The novel methodology produced a plot of four vertically aligned graphs that enables decision-makers to identify an alert threshold that optimally balances effectiveness, efficiency and utility (EEU) at the level of an entire admission, and which differs from that derived using conventional metrics.

Discussion: Conventional evaluation metrics do not consider alert timing relative to clinical events and are often applied to different evaluation datasets (sample and admission level), introducing bias and confusion. In contrast, the EEU methodology (i) generates admission level evaluations at different alert thresholds; (ii) measures alert timing relative to clinical events; and (iii) provides a visual display that enables identification of the alert threshold that optimally balances EEU factors.

Conclusion: Evaluations of prediction models for adverse events in hospitalised patients should incorporate the EEU approach in assessing model suitability and selecting alert thresholds.

Objective: To guide research data services (RDS) teams in managing researcher variability (eg, differing deadlines, funding, expertise) when honest-brokering data, we present a framework based on operations management principles and a food service analogy.

Materials and methods: Our framework describes 4 data service offerings with different levels of efficiency and service customization: vending machine, fast food, custom meal, and personal chef. We describe examples from 2 institutions.

Results: Vending machine and fast food are efficient but less customizable, making them better-suited for researchers with limited funding or time. Custom meal and personal chef are less efficient but more customized, making them well suited for better-resourced researchers.

Discussion: Efficiency and service tradeoffs should be balanced to align with demand and institutional goals. RDS teams can overcome such tradeoffs through uncompromised reduction or low-cost accommodation approaches.

Conclusion: Our framework can be applied by RDS teams in their design and implementation of data services.

Objective: To build natural language processing (NLP) strategies to characterize measures of genetic counseling (GC) efficiency and classify measures according to phase of GC (pre- or post-genetic testing).

Materials and methods: This study selected and annotated 800 GC notes from 7 clinical specialties in a large academic medical center for NLP model development and validation. The NLP approaches extracted GC efficiency measures, including direct and indirect time and GC phase. The models were then applied to 24 102 GC notes collected from January 2016 through December 2023.

Results: NLP approaches performed well (F1 scores of 0.95 and 0.90 for direct time in GC and GC phase classification, respectively). Our findings showed median direct time in GC of 50 minutes, with significant differences in direct time distributions observed across clinical specialties, time periods (2016-2019 or 2020-2023), delivery modes (in person or telehealth), and GC phase.

Discussion: As referrals to GC increase, there is increasing pressure to improve efficiency. Our NLP strategy was used to generate and summarize real-world evidence of GC time for 7 clinical specialties. These approaches enable future research on the impact of interventions intended to improve GC efficiency.

Conclusion: This work demonstrated the practical value of NLP to provide a useful and scalable strategy to generate real world evidence of GC efficiency. Principles presented in this work may also be valuable for health services research in other practice areas.

Objective: Stigmatizing language (SL) in Electronic Health Records (EHRs) can perpetuate biases and negatively impact patient care. This study introduces a novel method for automatically detecting such language to improve healthcare documentation practices.

Materials and methods: We developed a multi-stage transfer learning framework integrating semantic, syntactic, and task adaptation using three datasets: hate speech, clinical phenotypes, and stigmatizing language. Experiments were conducted on stigmatizing language dataset which consists of 4,129 de-identified EHR notes (72.7% stigmatizing, 27.3% non-stigmatizing), split 80/20 for training and testing. Longformer, BERT, and ClinicalBERT models were evaluated, and model performance was assessed on 35 randomized subsets of the test set (each comprising 70% of test data). The Wilcoxon-Mann-Whitney test was used to evaluate statistical significance, with Bonferroni correction applied to control for multiple hypothesis testing. Baseline models included zero-shot and few-shot GPT-4o, Support Vector Machine, Random Forest, Logistic Regression, and Multinomial Naive Bayes.

Results: The proposed framework achieved the highest accuracy, with fully adapted Longformer reaching 89.83%. Performance improvements remained statistically significant after Bonferroni correction compared to all baselines (p < .05). The framework demonstrated robust gains across different stigmatizing language types.

Discussion: This study underscores the value of domain-adaptive NLP for detecting stigmatizing language in EHRs. The multi-stage transfer learning framework effectively captures subtle biases often missed by conventional models, enabling more objective and respectful clinical documentation.

Conclusion: This framework offers a statistically validated, high-performing framework for detecting stigmatizing language in EHRs, supporting responsible AI and promoting equity in clinical care.

Objective: Electronic health records (EHRs) lack a widely adopted standard for recording transgender and gender diverse (TGD) status, complicating research on TGD health. Computational models have been developed to identify TGD individuals in EHRs; however, gaps remain in understanding which components contribute to stronger phenotyping approaches. This scoping review evaluates EHR-based models for identifying TGD individuals, focusing on identifier types, performance, external validation, and ethical reporting to guide best practices.

Materials and methods: We searched PubMed, CINAHL, Web of Science, and Embase for peer-reviewed articles published before January 2024, following PRISMA-ScR guidelines. Included studies used EHR data to identify TGD individuals, verified TGD status, reported or allowed calculation of positive predictive value (PPV), and listed identifiers. Two authors screened and extracted data. We categorized models by data type and logic (structured, unstructured, and multimodal), summarized PPV distributions, and synthesized author-reported ethical considerations.

Results: Fourteen studies describing 50 models met inclusion criteria. Models using TGD-related diagnostic codes alone (n = 11) or requiring both structured and unstructured data (n = 6) showed the highest mean PPVs (85.3% and 97.1%). Models validated on larger confirmed TGD cohorts reported more stable performance, but external validation was rare. Most studies minimally addressed ethics; only 3 described protective measures or stakeholder engagement.

Discussion: Phenotyping of TGD individuals in EHR data remains heterogeneous in design and ethical transparency. Reported PPVs should be interpreted cautiously, as performance is influenced by study design, sample size, and verification methods.

Conclusions: Our recommendations emphasize the components that strengthen phenotyping approaches-identifier choice, multimodal intersection logic, validation practices, and ethical safeguards-rather than endorsing any single model.