Journal of the American Medical Informatics Association最新文献

Objectives: Phenotyping is a core task in observational health research utilizing electronic health records (EHRs). Developing an accurate algorithm demands substantial input from domain experts, involving extensive literature review and evidence synthesis. This burdensome process limits scalability and delays knowledge discovery. We investigate the potential for leveraging large language models (LLMs) to enhance the efficiency of EHR phenotyping by generating high-quality algorithm drafts.

Materials and methods: We prompted four LLMs-GPT-4 and GPT-3.5 of ChatGPT, Claude 2, and Bard-in October 2023, asking them to generate executable phenotyping algorithms in the form of SQL queries adhering to a common data model (CDM) for three phenotypes (ie, type 2 diabetes mellitus, dementia, and hypothyroidism). Three phenotyping experts evaluated the returned algorithms across several critical metrics. We further implemented the top-rated algorithms and compared them against clinician-validated phenotyping algorithms from the Electronic Medical Records and Genomics (eMERGE) network.

Results: GPT-4 and GPT-3.5 exhibited significantly higher overall expert evaluation scores in instruction following, algorithmic logic, and SQL executability, when compared to Claude 2 and Bard. Although GPT-4 and GPT-3.5 effectively identified relevant clinical concepts, they exhibited immature capability in organizing phenotyping criteria with the proper logic, leading to phenotyping algorithms that were either excessively restrictive (with low recall) or overly broad (with low positive predictive values).

Conclusion: GPT versions 3.5 and 4 are capable of drafting phenotyping algorithms by identifying relevant clinical criteria aligned with a CDM. However, expertise in informatics and clinical experience is still required to assess and further refine generated algorithms.

Objectives: Integration of social determinants of health into health outcomes research will allow researchers to study health inequities. The All of Us Research Program has the potential to be a rich source of social determinants of health data. However, user-friendly recommendations for scoring and interpreting the All of Us Social Determinants of Health Survey are needed to return value to communities through advancing researcher competencies in use of the All of Us Research Hub Researcher Workbench. We created a user guide aimed at providing researchers with an overview of the Social Determinants of Health Survey, recommendations for scoring and interpreting participant responses, and readily executable R and Python functions.

Target audience: This user guide targets registered users of the All of Us Research Hub Researcher Workbench, a cloud-based platform that supports analysis of All of Us data, who are currently conducting or planning to conduct analyses using the Social Determinants of Health Survey.

Scope: We introduce 14 constructs evaluated as part of the Social Determinants of Health Survey and summarize construct operationalization. We offer 30 literature-informed recommendations for scoring participant responses and interpreting scores, with multiple options available for 8 of the constructs. Then, we walk through example R and Python functions for relabeling responses and scoring constructs that can be directly implemented in Jupyter Notebook or RStudio within the Researcher Workbench. Full source code is available in supplemental files and GitHub. Finally, we discuss psychometric considerations related to the Social Determinants of Health Survey for researchers.

Background: Hypertension (HTN) remains a significant public health concern and the primary modifiable risk factor for cardiovascular disease, which is the leading cause of death in the United States. We applied our validated HTN computable phenotypes within the All of Us Research Program to uncover prevalence and characteristics of HTN and apparent treatment-resistant hypertension (aTRH) in United States.

Methods: Within the All of Us Researcher Workbench, we built a retrospective cohort (January 1, 2008-July 1, 2023), identifying all adults with available age data, at least one blood pressure (BP) measurement, prescribed at least one antihypertensive medication, and with at least one SNOMED "Essential hypertension" diagnosis code.

Results: We identified 99 461 participants with HTN who met the eligibility criteria. Following the application of our computable phenotypes, an overall population of 81 462 were further categorized to aTRH (14.4%), stable-controlled HTN (SCH) (39.5%), and Other HTN (46.1%). Compared to participants with SCH, participants with aTRH were older, more likely to be of Black or African American race, had higher levels of social deprivation, and a heightened prevalence of comorbidities such as hyperlipidemia and diabetes. Heart failure, chronic kidney disease, and diabetes were the comorbidities most strongly associated with aTRH. β-blockers were the most prescribed antihypertensive medication. At index date, the overall BP control rate was 62%.

Discussion and conclusion: All of Us provides a unique opportunity to characterize HTN in the United States. Consistent findings from this study with our prior research highlight the interoperability of our computable phenotypes.

Objectives: Research participants value learning how their data contributions are advancing health research (ie, data stories). The All of Us Research Program gathered insights from program staff to learn what research topics they think are of interest to participants, what support staff need to communicate data stories, and how staff use data story dissemination tools.

Materials and methods: Using an online 25-item assessment, we collected information from All of Us staff at 7 Federally Qualified Health Centers.

Results: Topics of greatest interest or relevance included income insecurity (83%), diabetes (78%), and mental health (78%). Respondents prioritized in-person outreach in the community (70%) as a preferred setting to share data stories. Familiarity with available dissemination tools varied.

Discussion: Responses support prioritizing materials for in-person outreach and training staff how to use dissemination tools.

Conclusion: The findings will inform All of Us communication strategy, content, materials, and staff training resources to effectively deliver data stories as return of value to participants.

Objectives: Mobile health (mHealth) regimens can improve health through the continuous monitoring of biometric parameters paired with appropriate interventions. However, adherence to monitoring tends to decay over time. Our randomized controlled trial sought to determine: (1) if a mobile app with gamification and financial incentives significantly increases adherence to mHealth monitoring in a population of heart failure patients; and (2) if activity data correlate with disease-specific symptoms.

Materials and methods: We recruited individuals with heart failure into a prospective 180-day monitoring study with 3 arms. All 3 arms included monitoring with a connected weight scale and an activity tracker. The second arm included an additional mobile app with gamification, and the third arm included the mobile app and a financial incentive awarded based on adherence to mobile monitoring.

Results: We recruited 111 heart failure patients into the study. We found that the arm including the financial incentive led to significantly higher adherence to activity tracker (95% vs 72.2%, P = .01) and weight (87.5% vs 69.4%, P = .002) monitoring compared to the arm that included the monitoring devices alone. Furthermore, we found a significant correlation between daily steps and daily symptom severity.

Discussion and conclusion: Our findings indicate that mobile apps with added engagement features can be useful tools for improving adherence over time and may thus increase the impact of mHealth-driven interventions. Additionally, activity tracker data can provide passive monitoring of disease burden that may be used to predict future events.

Objectives: We describe the development and implementation of a system for monitoring patient-reported adverse events and quality of life using electronic Patient Reported Outcome (ePRO) instruments in the I-SPY2 Trial, a phase II clinical trial for locally advanced breast cancer. We describe the administration of technological, workflow, and behavior change interventions and their associated impact on questionnaire completion.

Materials and methods: Using the OpenClinica electronic data capture system, we developed rules-based logic to build automated ePRO surveys, customized to the I-SPY2 treatment schedule. We piloted ePROs at the University of California, San Francisco (UCSF) to optimize workflow in the context of trial treatment scenarios and staggered rollout of the ePRO system to 26 sites to ensure effective implementation of the technology.

Results: Increasing ePRO completion requires workflow solutions and research staff engagement. Over two years, we increased baseline survey completion from 25% to 80%. The majority of patients completed between 30% and 75% of the questionnaires they received, with no statistically significant variation in survey completion by age, race or ethnicity. Patients who completed the screening timepoint questionnaire were significantly more likely to complete more of the surveys they received at later timepoints (mean completion of 74.1% vs 35.5%, P < .0001). Baseline PROMIS social functioning and grade 2 or more PRO-CTCAE interference of Abdominal Pain, Decreased Appetite, Dizziness and Shortness of Breath was associated with lower survey completion rates.

Discussion and conclusion: By implementing ePROs, we have the potential to increase efficiency and accuracy of patient-reported clinical trial data collection, while improving quality of care, patient safety, and health outcomes. Our method is accessible across demographics and facilitates an ease of data collection and sharing across nationwide sites. We identify predictors of decreased completion that can optimize resource allocation by better targeting efforts such as in-person outreach, staff engagement, a robust technical workflow, and increased monitoring to improve overall completion rates.

Trial registration: https://clinicaltrials.gov/study/NCT01042379.

Importance: Scales often arise from multi-item questionnaires, yet commonly face item non-response. Traditional solutions use weighted mean (WMean) from available responses, but potentially overlook missing data intricacies. Advanced methods like multiple imputation (MI) address broader missing data, but demand increased computational resources. Researchers frequently use survey data in the All of Us Research Program (All of Us), and it is imperative to determine if the increased computational burden of employing MI to handle non-response is justifiable.

Objectives: Using the 5-item Physical Activity Neighborhood Environment Scale (PANES) in All of Us, this study assessed the tradeoff between efficacy and computational demands of WMean, MI, and inverse probability weighting (IPW) when dealing with item non-response.

Materials and methods: Synthetic missingness, allowing 1 or more item non-response, was introduced into PANES across 3 missing mechanisms and various missing percentages (10%-50%). Each scenario compared WMean of complete questions, MI, and IPW on bias, variability, coverage probability, and computation time.

Results: All methods showed minimal biases (all <5.5%) for good internal consistency, with WMean suffered most with poor consistency. IPW showed considerable variability with increasing missing percentage. MI required significantly more computational resources, taking >8000 and >100 times longer than WMean and IPW in full data analysis, respectively.

Discussion and conclusion: The marginal performance advantages of MI for item non-response in highly reliable scales do not warrant its escalated cloud computational burden in All of Us, particularly when coupled with computationally demanding post-imputation analyses. Researchers using survey scales with low missingness could utilize WMean to reduce computing burden.

Objectives: The All of Us Research Program is a precision medicine initiative aimed at establishing a vast, diverse biomedical database accessible through a cloud-based data analysis platform, the Researcher Workbench (RW). Our goal was to empower the research community by co-designing the implementation of SAS in the RW alongside researchers to enable broader use of All of Us data.

Materials and methods: Researchers from various fields and with different SAS experience levels participated in co-designing the SAS implementation through user experience interviews.

Results: Feedback and lessons learned from user testing informed the final design of the SAS application.

Discussion: The co-design approach is critical for reducing technical barriers, broadening All of Us data use, and enhancing the user experience for data analysis on the RW.

Conclusion: Our co-design approach successfully tailored the implementation of the SAS application to researchers' needs. This approach may inform future software implementations on the RW.

Objective: This article outlines a scalable system developed by the All of Us Research Program's Genetic Counseling Resource to vet a large database of healthcare resources for supporting participants with health-related DNA results.

Materials and methods: After a literature review of established evaluation frameworks for health resources, we created SONAR, a 10-item framework and grading scale for health-related participant-facing resources. SONAR was used to review clinical resources that could be shared with participants during genetic counseling.

Results: Application of SONAR shortened resource approval time from 7 days to 1 day. About 256 resources were approved and 8 rejected through SONAR review. Most approved resources were relevant to participants nationwide (60.0%). The most common resource types were related to support groups (20%), cancer care (30.6%), and general educational resources (12.4%). All of Us genetic counselors provided 1161 approved resources during 3005 (38.6%) consults, mainly to local genetic counselors (29.9%), support groups (21.9%), and educational resources (21.0%).

Discussion: SONAR's systematic method simplifies resource vetting for healthcare providers, easing the burden of identifying and evaluating credible resources. Compiling these resources into a user-friendly database allows providers to share these resources efficiently, better equipping participants to complete follow up actions from health-related DNA results.

Conclusion: The All of Us Genetic Counseling Resource connects participants receiving health-related DNA results with relevant follow-up resources on a high-volume, national level. This has been made possible by the creation of a novel resource database and validation system.