Journal of the American Medical Informatics Association最新文献

Objective: This study leverages the rich diversity of the All of Us Research Program (All of Us)'s dataset to devise a predictive model for cardiovascular disease (CVD) in breast cancer (BC) survivors. Central to this endeavor is the creation of a robust data integration pipeline that synthesizes electronic health records (EHRs), patient surveys, and genomic data, while upholding fairness across demographic variables.

Materials and methods: We have developed a universal data wrangling pipeline to process and merge heterogeneous data sources of the All of Us dataset, address missingness and variance in data, and align disparate data modalities into a coherent framework for analysis. Utilizing a composite feature set including EHR, lifestyle, and social determinants of health (SDoH) data, we then employed Adaptive Lasso and Random Forest regression models to predict 6 CVD outcomes. The models were evaluated using the c-index and time-dependent Area Under the Receiver Operating Characteristic Curve over a 10-year period.

Results: The Adaptive Lasso model showed consistent performance across most CVD outcomes, while the Random Forest model excelled particularly in predicting outcomes like transient ischemic attack when incorporating the full multi-model feature set. Feature importance analysis revealed age and previous coronary events as dominant predictors across CVD outcomes, with SDoH clustering labels highlighting the nuanced impact of social factors.

Discussion: The development of both Cox-based predictive model and Random Forest Regression model represents the extensive application of the All of Us, in integrating EHR and patient surveys to enhance precision medicine. And the inclusion of SDoH clustering labels revealed the significant impact of sociobehavioral factors on patient outcomes, emphasizing the importance of comprehensive health determinants in predictive models. Despite these advancements, limitations include the exclusion of genetic data, broad categorization of CVD conditions, and the need for fairness analyses to ensure equitable model performance across diverse populations. Future work should refine clinical and social variable measurements, incorporate advanced imputation techniques, and explore additional predictive algorithms to enhance model precision and fairness.

Conclusion: This study demonstrates the liability of the All of Us's diverse dataset in developing a multi-modality predictive model for CVD in BC survivors risk stratification in oncological survivorship. The data integration pipeline and subsequent predictive models establish a methodological foundation for future research into personalized healthcare.

Objectives: To evaluate the NIH All of Us Research Program database as a potential data source for studying allostatic load and stress among adults in the United States (US).

Materials and methods: We evaluated the All of Us database to determine sample size significance for original-10 allostatic load biomarkers, Allostatic Load Index-5 (ALI-5), Allostatic Load Five, and Cohen's Perceived Stress Scale (PSS). We conducted a priori, post hoc, and sensitivity power analyses to determine sample sizes for conducting null hypothesis significance tests.

Results: The maximum number of responses available for each measure is 21 participants for the original-10 allostatic load biomarkers, 150 for the ALI-5, 22 476 for Allostatic Load Five, and n = 90 583 for the PSS.

Discussion: The NIH All of Us Research Program is well-suited for studying allostatic load using the Allostatic Load Five and psychological stress using PSS.

Conclusion: Improving biomarker data collection in All of Us will facilitate more nuanced examinations of allostatic load among US adults.

Objective: Summaries of health research can be a complementary way to return value to participants. We assess how research participants engage with summaries via email communication and how this can be improved.

Materials and methods: We look at correlations between demographic subgroups and engagement in a longitudinal dataset of 305 626 participants (77% are classified as underrepresented in biomedical research) from the All of Us Research Program. We compare this against engagement with other program communications and use impact evaluations (N = 421 510) to measure the effect of tailoring communication by (1) eliciting content preferences, (2) Spanish focused content, (3) informational videos, and (4) article content in the email subject line.

Results: Between March 2020 and October 2021, research summaries reached 67% of enrolled participants, outperforming other program communication (60%) and return of results (31%), which have a high uptake rate but have been extended to a subset of eligible participants. While all demographic subgroups engage with research summaries, participants with higher income, educational attainment, White, and older than 45 years open and click content most often. Surfacing article content in the email subject line and Spanish focused content had negative effects on engagement. Video and social media content and eliciting preferences led to a small directional increase in clicks.

Discussion: Further individualization of tailoring efforts may be needed to drive larger engagement effects (eg, delivering multiple articles in line with stated preferences, expanding preference options). Our findings are likely a conservative representation of engagement effects, given the coarseness of our click rate measure.

Conclusions: Health research summaries show promise as a way to return value to research participants, especially if individual-level results cannot be returned. Personalization of communication requires testing to determine whether efforts are having the expected effect.

Objectives: To understand the landscape of privacy preserving record linkage (PPRL) applications in public health, assess estimates of PPRL accuracy and privacy, and evaluate factors for PPRL adoption.

Materials and methods: A literature scan examined the accuracy, data privacy, and scalability of PPRL in public health. Twelve interviews with subject matter experts were conducted and coded using an inductive approach to identify factors related to PPRL adoption.

Results: PPRL has a high level of linkage quality and accuracy. PPRL linkage quality was comparable to that of clear text linkage methods (requiring direct personally identifiable information [PII]) for linkage across various settings and research questions. Accuracy of PPRL depended on several components, such as PPRL technique, and the proportion of missingness and errors in underlying data. Strategies to increase adoption include increasing understanding of PPRL, improving data owner buy-in, establishing governance structure and oversight, and developing a public health implementation strategy for PPRL.

Discussion: PPRL protects privacy by eliminating the need to share PII for linkage, but the accuracy and linkage quality depend on factors including the choice of PPRL technique and specific PII used to create encrypted identifiers. Large-scale implementations of PPRL linking millions of observations-including PCORnet, National Institutes for Health N3C, and the Centers for Disease Control and Prevention COVID-19 project have demonstrated the scalability of PPRL for public health applications.

Conclusions: Applications of PPRL in public health have demonstrated their value for the public health community. Although gaps must be addressed before wide implementation, PPRL is a promising solution to data linkage challenges faced by the public health ecosystem.

Objectives: To enhance and evaluate the quality of PubMed search results for Social Determinants of Health (SDoH) through the addition of new SDoH terms to Medical Subject Headings (MeSH).

Materials and methods: High priority SDoH terms and definitions were collated from authoritative sources, curated based on publication frequencies, and refined by subject matter experts. Descriptive analyses were used to investigate how PubMed search details and best match results were affected by the addition of SDoH concepts to MeSH. Three information retrieval metrics (Precision, Recall, and F measure) were used to quantitatively assess the accuracy of PubMed search results. Pre- and post-update documents were clustered into topic areas using a Natural Language Processing pipeline, and SDoH relevancy assessed.

Results: Addition of 35 SDoH terms to MeSH resulted in more accurate algorithmic translations of search terms and more reliable best match results. The Precision, Recall, and F measures of post-update results were significantly higher than those of pre-update results. The percentage of retrieved publications belonging to SDoH clusters was significantly greater in the post- than pre-update searches.

Discussion: This evaluation confirms that inclusion of new SDoH terms in MeSH can lead to qualitative and quantitative enhancements in PubMed search retrievals. It demonstrates the methodology for and impact of suggesting new terms for MeSH indexing. It provides a foundation for future efforts across behavioral and social science research (BSSR) domains.

Conclusion: Improving the representation of BSSR terminology in MeSH can improve PubMed search results, thereby enhancing the ability of investigators and clinicians to build and utilize a cumulative BSSR knowledge base.

Objectives: Despite easy-to-use tools like the Cohort Builder, using All of Us Research Program data for complex research questions requires a relatively high level of technical expertise. We aimed to increase research and training capacity and reduce barriers to entry for the All of Us community through an R package, allofus. In this article, we describe functions that address common challenges we encountered while working with All of Us Research Program data, and we demonstrate this functionality with an example of creating a cohort of All of Us participants by synthesizing electronic health record and survey data with time dependencies.

Target audience: All of Us Research Program data are widely available to health researchers. The allofus R package is aimed at a wide range of researchers who wish to conduct complex analyses using best practices for reproducibility and transparency, and who have a range of experience using R. Because the All of Us data are transformed into the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM), researchers familiar with existing OMOP CDM tools or who wish to conduct network studies in conjunction with other OMOP CDM data will also find value in the package.

Scope: We developed an initial set of functions that solve problems we experienced across survey and electronic health record data in our own research and in mentoring student projects. The package will continue to grow and develop with the All of Us Research Program. The allofus R package can help build community research capacity by increasing access to the All of Us Research Program data, the efficiency of its use, and the rigor and reproducibility of the resulting research.

Objectives: The National Institutes of Health's All of Us Research Program addresses gaps in biomedical research by collecting health data from diverse populations. Pregnant individuals have historically been underrepresented in biomedical research, and pregnancy-related research is often limited by data availability, sample size, and inadequate representation of the diversity of pregnant people. All of Us integrates a wealth of health-related data, providing a unique opportunity to conduct comprehensive pregnancy-related research. We aimed to identify pregnancy episodes with high-quality electronic health record (EHR) data in All of Us Research Program data and evaluate the program's utility for pregnancy-related research.

Materials and methods: We used a previously published algorithm to identify pregnancy episodes in All of Us EHR data. We described these pregnancies, validated them with All of Us survey data, and compared them to national statistics.

Results: Our study identified 18 970 pregnancy episodes from 14 234 participants; other possible pregnancy episodes had low-quality or insufficient data. Validation against people who reported a current pregnancy on an All of Us survey found low false positive and negative rates. Demographics were similar in some respects to national data; however, Asian-Americans were underrepresented, and older, highly educated pregnant people were overrepresented.

Discussion: Our approach demonstrates the capacity of All of Us to support pregnancy research and reveals the diversity of the pregnancy cohort. However, we noted an underrepresentation among some demographics. Other limitations include measurement error in gestational age and limited data on non-live births.

Conclusion: The wide variety of data in the All of Us program, encompassing EHR, survey, genomic, and fitness tracker data, offers a valuable resource for studying pregnancy, yet care must be taken to avoid biases.

Objectives: Outpatient rehabilitation (rehab) physical, occupational, and speech therapists use electronic health records (EHR), yet their documentation experiences, including any documentation burden, are not well researched. Therapists are a growing portion of the U.S. healthcare workforce, whose need is critical to the health of an aging population. We aimed to describe outpatient rehab therapists' documentation experiences and identify strategies for mitigating any documentation burden.

Materials and methods: We used qualitative descriptive methodology to conduct 4 focus groups with outpatient rehab therapists at Hospital for Special Surgery, a multi-site orthopedic institution. Transcripts were inductively coded to identify themes and actionable strategies for improving the therapists' documentation experiences. Therapists provided feedback and prioritization of proposed strategies.

Results: A total of 13 therapists were interviewed. Five themes and 10 subthemes characterize the therapists' documentation experience by a feeling that documentation inhibits clinical care and work/life balance, a perceived lack of support and efficiencies, the desire to document to communicate clinical care, and a design vision for improving the EHR. Top prioritized strategies for improvement included use of timesaving templates, expanding dictation, decluttering the EHR interface, and support for free texting over discrete data capture.

Discussion: Outpatient rehab therapists experience documentation burden similar to that documented of physicians and nurses. Manual data entry imposes burden on therapists' time and clinical care.

Conclusion: A multi-faceted approach is needed for improving therapists' experiences including EHR redesign, technology supporting dictation and narrative to discrete data capture, and support from leadership and regulators.

Objectives: Large Language Models (LLMs) have been proposed as a solution to address high volumes of Patient Medical Advice Requests (PMARs). This study addresses whether LLMs can generate high quality draft responses to PMARs that satisfies both patients and clinicians with prompt engineering.

Materials and methods: We designed a novel human-involved iterative processes to train and validate prompts to LLM in creating appropriate responses to PMARs. GPT-4 was used to generate response to the messages. We updated the prompts, and evaluated both clinician and patient acceptance of LLM-generated draft responses at each iteration, and tested the optimized prompt on independent validation data sets. The optimized prompt was implemented in the electronic health record production environment and tested by 69 primary care clinicians.

Results: After 3 iterations of prompt engineering, physician acceptance of draft suitability increased from 62% to 84% (P <.001) in the validation dataset (N = 200), and 74% of drafts in the test dataset were rated as "helpful." Patients also noted significantly increased favorability of message tone (78%) and overall quality (80%) for the optimized prompt compared to the original prompt in the training dataset, patients were unable to differentiate human and LLM-generated draft PMAR responses for 76% of the messages, in contrast to the earlier preference for human-generated responses. Majority (72%) of clinicians believed it can reduce cognitive load in dealing with InBasket messages.

Discussion and conclusion: Informed by clinician and patient feedback synergistically, tuning in LLM prompt alone can be effective in creating clinically relevant and useful draft responses to PMARs.

Objectives: This work presents the development and evaluation of coordn8, a web-based application that streamlines fax processing in outpatient clinics using a "human-in-the-loop" machine learning framework. We demonstrate the effectiveness of the platform at reducing fax processing time and producing accurate machine learning inferences across the tasks of patient identification, document classification, spam classification, and duplicate document detection.

Methods: We deployed coordn8 in 11 outpatient clinics and conducted a time savings analysis by observing users and measuring fax processing event logs. We used statistical methods to evaluate the machine learning components across different datasets to show generalizability. We conducted a time series analysis to show variations in model performance as new clinics were onboarded and to demonstrate our approach to mitigating model drift.

Results: Our observation analysis showed a mean reduction in individual fax processing time by 147.5 s, while our event log analysis of over 7000 faxes reinforced this finding. Document classification produced an accuracy of 81.6%, patient identification produced an accuracy of 83.7%, spam classification produced an accuracy of 98.4%, and duplicate document detection produced a precision of 81.0%. Retraining document classification increased accuracy by 10.2%.

Discussion: coordn8 significantly decreased fax-processing time and produced accurate machine learning inferences. Our human-in-the-loop framework facilitated the collection of high-quality data necessary for model training. Expanding to new clinics correlated with performance decline, which was mitigated through model retraining.

Conclusion: Our framework for automating clinical tasks with machine learning offers a template for health systems looking to implement similar technologies.