Journal of the American Medical Informatics Association最新文献

Objectives: To evaluate the NIH All of Us Research Program database as a potential data source for studying allostatic load and stress among adults in the United States (US).

Materials and methods: We evaluated the All of Us database to determine sample size significance for original-10 allostatic load biomarkers, Allostatic Load Index-5 (ALI-5), Allostatic Load Five, and Cohen's Perceived Stress Scale (PSS). We conducted a priori, post hoc, and sensitivity power analyses to determine sample sizes for conducting null hypothesis significance tests.

Results: The maximum number of responses available for each measure is 21 participants for the original-10 allostatic load biomarkers, 150 for the ALI-5, 22 476 for Allostatic Load Five, and n = 90 583 for the PSS.

Discussion: The NIH All of Us Research Program is well-suited for studying allostatic load using the Allostatic Load Five and psychological stress using PSS.

Conclusion: Improving biomarker data collection in All of Us will facilitate more nuanced examinations of allostatic load among US adults.

Objective: Summaries of health research can be a complementary way to return value to participants. We assess how research participants engage with summaries via email communication and how this can be improved.

Materials and methods: We look at correlations between demographic subgroups and engagement in a longitudinal dataset of 305 626 participants (77% are classified as underrepresented in biomedical research) from the All of Us Research Program. We compare this against engagement with other program communications and use impact evaluations (N = 421 510) to measure the effect of tailoring communication by (1) eliciting content preferences, (2) Spanish focused content, (3) informational videos, and (4) article content in the email subject line.

Results: Between March 2020 and October 2021, research summaries reached 67% of enrolled participants, outperforming other program communication (60%) and return of results (31%), which have a high uptake rate but have been extended to a subset of eligible participants. While all demographic subgroups engage with research summaries, participants with higher income, educational attainment, White, and older than 45 years open and click content most often. Surfacing article content in the email subject line and Spanish focused content had negative effects on engagement. Video and social media content and eliciting preferences led to a small directional increase in clicks.

Discussion: Further individualization of tailoring efforts may be needed to drive larger engagement effects (eg, delivering multiple articles in line with stated preferences, expanding preference options). Our findings are likely a conservative representation of engagement effects, given the coarseness of our click rate measure.

Conclusions: Health research summaries show promise as a way to return value to research participants, especially if individual-level results cannot be returned. Personalization of communication requires testing to determine whether efforts are having the expected effect.

Objectives: Despite easy-to-use tools like the Cohort Builder, using All of Us Research Program data for complex research questions requires a relatively high level of technical expertise. We aimed to increase research and training capacity and reduce barriers to entry for the All of Us community through an R package, allofus. In this article, we describe functions that address common challenges we encountered while working with All of Us Research Program data, and we demonstrate this functionality with an example of creating a cohort of All of Us participants by synthesizing electronic health record and survey data with time dependencies.

Target audience: All of Us Research Program data are widely available to health researchers. The allofus R package is aimed at a wide range of researchers who wish to conduct complex analyses using best practices for reproducibility and transparency, and who have a range of experience using R. Because the All of Us data are transformed into the Observational Medical Outcomes Partnership Common Data Model (OMOP CDM), researchers familiar with existing OMOP CDM tools or who wish to conduct network studies in conjunction with other OMOP CDM data will also find value in the package.

Scope: We developed an initial set of functions that solve problems we experienced across survey and electronic health record data in our own research and in mentoring student projects. The package will continue to grow and develop with the All of Us Research Program. The allofus R package can help build community research capacity by increasing access to the All of Us Research Program data, the efficiency of its use, and the rigor and reproducibility of the resulting research.

Objectives: The National Institutes of Health's All of Us Research Program addresses gaps in biomedical research by collecting health data from diverse populations. Pregnant individuals have historically been underrepresented in biomedical research, and pregnancy-related research is often limited by data availability, sample size, and inadequate representation of the diversity of pregnant people. All of Us integrates a wealth of health-related data, providing a unique opportunity to conduct comprehensive pregnancy-related research. We aimed to identify pregnancy episodes with high-quality electronic health record (EHR) data in All of Us Research Program data and evaluate the program's utility for pregnancy-related research.

Materials and methods: We used a previously published algorithm to identify pregnancy episodes in All of Us EHR data. We described these pregnancies, validated them with All of Us survey data, and compared them to national statistics.

Results: Our study identified 18 970 pregnancy episodes from 14 234 participants; other possible pregnancy episodes had low-quality or insufficient data. Validation against people who reported a current pregnancy on an All of Us survey found low false positive and negative rates. Demographics were similar in some respects to national data; however, Asian-Americans were underrepresented, and older, highly educated pregnant people were overrepresented.

Discussion: Our approach demonstrates the capacity of All of Us to support pregnancy research and reveals the diversity of the pregnancy cohort. However, we noted an underrepresentation among some demographics. Other limitations include measurement error in gestational age and limited data on non-live births.

Conclusion: The wide variety of data in the All of Us program, encompassing EHR, survey, genomic, and fitness tracker data, offers a valuable resource for studying pregnancy, yet care must be taken to avoid biases.

Objectives: We introduce a widely applicable model-based approach for estimating individual-level Social Determinants of Health (SDoH) and evaluate its effectiveness using the All of Us Research Program.

Materials and methods: Our approach utilizes aggregated SDoH datasets to estimate individual-level SDoH, demonstrated with examples of no high school diploma (NOHSDP) and no health insurance (UNINSUR) variables. Models are estimated using American Community Survey data and applied to derive individual-level estimates for All of Us participants. We assess concordance between model-based SDoH estimates and self-reported SDoHs in All of Us and examine associations with undiagnosed hypertension and diabetes.

Results: Compared to self-reported SDoHs, the area under the curve for NOHSDP is 0.727 (95% CI, 0.724-0.730) and for UNINSUR is 0.730 (95% CI, 0.727-0.733) among the 329 074 All of Us participants, both significantly higher than aggregated SDoHs. The association between model-based NOHSDP and undiagnosed hypertension is concordant with those estimated using self-reported NOHSDP, with a correlation coefficient of 0.649. Similarly, the association between model-based NOHSDP and undiagnosed diabetes is concordant with those estimated using self-reported NOHSDP, with a correlation coefficient of 0.900.

Discussion and conclusion: The model-based SDoH estimation method offers a scalable and easily standardized approach for estimating individual-level SDoHs. Using the All of Us dataset, we demonstrate reasonable concordance between model-based SDoH estimates and self-reported SDoHs, along with consistent associations with health outcomes. Our findings also underscore the critical role of geographic contexts in SDoH estimation and in evaluating the association between SDoHs and health outcomes.

Objectives: We describe new curriculum materials for engaging secondary school students in exploring the "big data" in the NIH All of Us Research Program's Public Data Browser and the co-design processes used to collaboratively develop the materials. We also describe the methods used to develop and validate assessment items for studying the efficacy of the materials for student learning as well as preliminary findings from these studies.

Materials and methods: Secondary-level biology teachers from across the United States participated in a 2.5-day Co-design Summer Institute. After learning about the All of Us Research Program and its Data Browser, they collaboratively developed learning objectives and initial ideas for learning experiences related to exploring the Data Browser and big data. The Genetic Science Learning Center team at the University of Utah further developed the educators' ideas. Additional teachers and their students participated in classroom pilot studies to validate a 22-item instrument that assesses students' knowledge. Educators completed surveys about the materials and their experiences.

Results: The "Exploring Big Data with the All of Us Data Browser" curriculum module includes 3 data exploration guides that engage students in using the Data Browser, 3 related multimedia pieces, and teacher support materials. Pilot testing showed substantial growth in students' understanding of key big data concepts and research applications.

Discussion and conclusion: Our co-design process provides a model for educator engagement. The new curriculum module serves as a model for introducing secondary students to big data and precision medicine research by exploring diverse real-world datasets.

Objectives: To highlight the use of calibration weighting to improve the precision of estimates obtained from All of Us data and increase the return of value to communities from the All of Us Research Program.

Materials and methods: We used All of Us (2017-2022) data and raking to obtain prevalence estimates in two examples: discrimination in medical settings (N = 41 875) and food insecurity (N = 82 266). Weights were constructed using known population proportions (age, sex, race/ethnicity, region of residence, annual household income, and home ownership) from the 2020 National Health Interview Survey.

Results: About 37% of adults experienced discrimination in a medical setting. About 20% of adults who had not seen a doctor reported being food insecure compared with 14% of adults who regularly saw a doctor.

Conclusions: Calibration using raking is cost-effective and may lead to more precise estimates when analyzing All of Us data.

Objective: This study aims to develop machine learning models that provide both accurate and equitable predictions of 2-year stroke risk for patients with atrial fibrillation across diverse racial groups.

Materials and methods: Our study utilized structured electronic health records (EHR) data from the All of Us Research Program. Machine learning models (LightGBM) were utilized to capture the relations between stroke risks and the predictors used by the widely recognized CHADS2 and CHA2DS2-VASc scores. We mitigated the racial disparity by creating a representative tuning set, customizing tuning criteria, and setting binary thresholds separately for subgroups. We constructed a hold-out test set that not only supports temporal validation but also includes a larger proportion of Black/African Americans for fairness validation.

Results: Compared to the original CHADS2 and CHA2DS2-VASc scores, significant improvements were achieved by modeling their predictors using machine learning models (Area Under the Receiver Operating Characteristic curve from near 0.70 to above 0.80). Furthermore, applying our disparity mitigation strategies can effectively enhance model fairness compared to the conventional cross-validation approach.

Discussion: Modeling CHADS2 and CHA2DS2-VASc risk factors with LightGBM and our disparity mitigation strategies achieved decent discriminative performance and excellent fairness performance. In addition, this approach can provide a complete interpretation of each predictor. These highlight its potential utility in clinical practice.

Conclusions: Our research presents a practical example of addressing clinical challenges through the All of Us Research Program data. The disparity mitigation framework we proposed is adaptable across various models and data modalities, demonstrating broad potential in clinical informatics.

Background: Acute hepatic porphyria (AHP) is a group of rare but treatable conditions associated with diagnostic delays of 15 years on average. The advent of electronic health records (EHR) data and machine learning (ML) may improve the timely recognition of rare diseases like AHP. However, prediction models can be difficult to train given the limited case numbers, unstructured EHR data, and selection biases intrinsic to healthcare delivery. We sought to train and characterize models for identifying patients with AHP.

Methods: This diagnostic study used structured and notes-based EHR data from 2 centers at the University of California, UCSF (2012-2022) and UCLA (2019-2022). The data were split into 2 cohorts (referral and diagnosis) and used to develop models that predict (1) who will be referred for testing of acute porphyria, among those who presented with abdominal pain (a cardinal symptom of AHP), and (2) who will test positive, among those referred. The referral cohort consisted of 747 patients referred for testing and 99 849 contemporaneous patients who were not. The diagnosis cohort consisted of 72 confirmed AHP cases and 347 patients who tested negative. The case cohort was 81% female and 6-75 years old at the time of diagnosis. Candidate models used a range of architectures. Feature selection was semi-automated and incorporated publicly available data from knowledge graphs. Our primary outcome was the F-score on an outcome-stratified test set.

Results: The best center-specific referral models achieved an F-score of 86%-91%. The best diagnosis model achieved an F-score of 92%. To further test our model, we contacted 372 current patients who lack an AHP diagnosis but were predicted by our models as potentially having it (≥10% probability of referral, ≥50% of testing positive). However, we were only able to recruit 10 of these patients for biochemical testing, all of whom were negative. Nonetheless, post hoc evaluations suggested that these models could identify 71% of cases earlier than their diagnosis date, saving 1.2 years.

Conclusions: ML can reduce diagnostic delays in AHP and other rare diseases. Robust recruitment strategies and multicenter coordination will be needed to validate these models before they can be deployed.

Objectives: The goal of this case report is to detail experiences and challenges experienced in the training of Primary Care residents in secondary analysis using All of Us Researcher Workbench. At our large, urban safety net hospital, Primary Care/Internal Medicine residents in their third year undergo a research intensive block, the Research Practicum, where they work as a team to conduct secondary data analysis on a dataset with faculty facilitation. In 2023, this research block focused on use of the All of Us Researcher Workbench for secondary data analysis.

Materials and methods: Two groups of 5 residents underwent training to access the All of Us Researcher Workbench, and each group explored available data with a faculty facilitator and generated original research questions. Two blocks of residents successfully completed their research blocks and created original presentations on "social isolation and A1C" levels and "medical discrimination and diabetes management."

Results: Departmental faculty were satisfied with the depth of learning and data exploration. In focus groups, some residents noted that for those without interest in performing research, the activity felt extraneous to their career goals, while others were glad for the opportunity to publish. In both blocks, residents highlighted dissatisfaction with the degree to which the All of Us Researcher Workbench was representative of patients they encounter in a large safety net hospital.

Discussion: Using the All of Us Researcher Workbench provided residents with an opportunity to explore novel questions in a massive data source. Many residents however noted that because the population described in the All of Us Researcher Workbench appeared to be more highly educated and less racially diverse than patients they encounter in their practice, research may be hard to generalize in a community health context. Additionally, given that the data required knowledge of 1 of 2 code-based data analysis languages (R or Python) and work within an idiosyncratic coding environment, residents were heavily reliant on a faculty facilitator to assist with analysis.

Conclusion: Using the All of Us Researcher Workbench for research training allowed residents to explore novel questions and gain first-hand exposure to opportunities and challenges in secondary data analysis.