Journal of the American Medical Informatics Association最新文献

Objective: To determine the efficacy of the mLab App, a mobile-delivered HIV prevention intervention to increase HIV self-testing in MSM and TGW.

Materials and methods: This was a randomized (2:2:1) clinical trial of the efficacy the mLab App as compared to standard of care vs mailed home HIV test arm among 525 MSM and TGW aged 18-29 years to increase HIV testing.

Results: The mLab App arm participants demonstrated an increase from 35.1% reporting HIV testing in the prior 6 months compared to 88.5% at 6 months. In contrast, 28.8% of control participants reported an HIV test at baseline, which only increased to 65.1% at 6 months. In a generalized linear mixed model estimating this change and controlling for multiple observations of participants, this equated to control participants reporting a 61.2% smaller increase in HIV testing relative to mLab participants (P = .001) at 6 months. This difference was maintained at 12 months with control participants reporting an 82.6% smaller increase relative to mLab App participants (P < .001) from baseline to 12 months.

Discussion and conclusion: Findings suggest that the mLab App is well-supported, evidence-based, behavioral risk-reduction intervention for increasing HIV testing rates as compared to the standard of care, suggesting that this may be a useful behavioral risk-reduction intervention for increasing HIV testing among young MSM.

Trial registration: This trial was registered with Clinicaltrials.gov NCT03803683.

Objectives: This study aims to improve the ethical use of machine learning (ML)-based clinical prediction models (CPMs) in shared decision-making for patients with kidney failure on dialysis. We explore factors that inform acceptability, interpretability, and implementation of ML-based CPMs among multiple constituent groups.

Materials and methods: We collected and analyzed qualitative data from focus groups with varied end users, including: dialysis support providers (clinical providers and additional dialysis support providers such as dialysis clinic staff and social workers); patients; patients' caregivers (n = 52).

Results: Participants were broadly accepting of ML-based CPMs, but with concerns on data sources, factors included in the model, and accuracy. Use was desired in conjunction with providers' views and explanations. Differences among respondent types were minimal overall but most prevalent in discussions of CPM presentation and model use.

Discussion and conclusion: Evidence of acceptability of ML-based CPM usage provides support for ethical use, but numerous specific considerations in acceptability, model construction, and model use for shared clinical decision-making must be considered. There are specific steps that could be taken by data scientists and health systems to engender use that is accepted by end users and facilitates trust, but there are also ongoing barriers or challenges in addressing desires for use. This study contributes to emerging literature on interpretability, mechanisms for sharing complexities, including uncertainty regarding the model results, and implications for decision-making. It examines numerous stakeholder groups including providers, patients, and caregivers to provide specific considerations that can influence health system use and provide a basis for future research.

Objectives: The NIH All of Us Research Program (All of Us) is engaging a diverse community of more than 10 000 registered researchers using a robust engagement ecosystem model. We describe strategies used to build an ecosystem that attracts and supports a diverse and inclusive researcher community to use the All of Us dataset and provide metrics on All of Us researcher usage growth.

Materials and methods: Researcher audiences and diversity categories were defined to guide a strategy. A researcher engagement strategy was codeveloped with program partners to support a researcher engagement ecosystem. An adapted ecological model guided the ecosystem to address multiple levels of influence to support All of Us data use. Statistics from the All of Us Researcher Workbench demographic survey describe trends in researchers' and institutional use of the Workbench and publication numbers.

Results: From 2022 to 2024, some 13 partner organizations and their subawardees conducted outreach, built capacity, or supported researchers and institutions in using the data. Trends indicate that Workbench registrations and use have increased over time, including among researchers underrepresented in the biomedical workforce. Data Use and Registration Agreements from minority-serving institutions also increased.

Discussion: All of Us built a diverse, inclusive, and growing research community via intentional engagement with researchers and via partnerships to address systemic data access issues. Future programs will provide additional support to researchers and institutions to ameliorate All of Us data use challenges.

Conclusion: The approach described helps address structural inequities in the biomedical research field to advance health equity.

Objective: To demonstrate the potential for a centrally managed health information exchange standardized to a common data model (HIE-CDM) to facilitate semantic data flow needed to support a learning health system (LHS).

Materials and methods: The Rhode Island Quality Institute operates the Rhode Island (RI) statewide HIE, which aggregates RI health data for more than half of the state's population from 47 data partners. We standardized HIE data to the Observational Medical Outcomes Partnership (OMOP) CDM. Atherosclerotic cardiovascular disease (ASCVD) risk and primary prevention practices were selected to demonstrate LHS semantic data flow from 2013 to 2023.

Results: We calculated longitudinal 10-year ASCVD risk on 62,999 individuals. Nearly two-thirds had ASCVD risk factors from more than one data partner. This enabled granular tracking of individual ASCVD risk, primary prevention (ie, statin therapy), and incident disease. The population was on statins for fewer than half of the guideline-recommended days. We also found that individuals receiving care at Federally Qualified Health Centers were more likely to have unfavorable ASCVD risk profiles and more likely to be on statins. CDM transformation reduced data heterogeneity through a unified health record that adheres to defined terminologies per OMOP domain.

Discussion: We demonstrated the potential for an HIE-CDM to enable observational population health research. We also showed how to leverage existing health information technology infrastructure and health data best practices to break down LHS barriers.

Conclusion: HIE-CDM facilitates knowledge curation and health system intervention development at the individual, health system, and population levels.

Objectives: Cancer diagnosis comes as a shock to many patients, and many of them feel unprepared to handle the complexity of the life-changing event, understand technicalities of the diagnostic reports, and fully engage with the clinical team regarding the personalized clinical decision-making.

Materials and methods: We develop Oncointerpreter.ai an interactive resource to offer personalized summarization of clinical cancer genomic and pathological data, and frame questions or address queries about therapeutic opportunities in near-real time via a graphical interface. It is built on the Mistral-7B and Llama-2 7B large language models trained on a local database trained using a large, curated corpus.

Results: We showcase its utility with case studies, where Oncointerpreter.ai extracted key clinical and molecular attributes from deidentified pathology and clinical genomics reports, summarized their contextual significance and answered queries on pertinent treatment options. Oncointerpreter also provided personalized summary of currently active clinical trials that match the patients' disease status, their selection criteria, and geographic locations. Benchmarking and comparative assessment indicated that the model responses were generally consistent, and hallucination, ie, factually incorrect or nonsensical response was rare; treatment- and outcome related queries led to context-aware responses, and response time correlated with verbosity.

Discussion: The choice of model and domain-specific training also affected the response quality.

Conclusion: Oncointerpreter.ai can aid the existing clinical care with interactive, individualized summarization of diagnostics data to promote informed dialogs with the patients with new cancer diagnoses.

Availability: https://github.com/Siris2314/Oncointerpreter.

Objective: Access to firearms is associated with increased suicide risk. Our aim was to develop a natural language processing approach to characterizing firearm access in clinical records.

Materials and methods: We used clinical notes from 36 685 Veterans Health Administration (VHA) patients between April 10, 2023 and April 10, 2024. We expanded preexisting firearm term sets using subject matter experts and generated 250-character snippets around each firearm term appearing in notes. Annotators labeled 3000 snippets into three classes. Using these annotated snippets, we compared four nonneural machine learning models (random forest, bagging, gradient boosting, logistic regression with ridge penalization) and two versions of Bidirectional Encoder Representations from Transformers, or BERT (specifically, BioBERT and Bio-ClinicalBERT) for classifying firearm access as "definite access", "definitely no access", or "other".

Results: Firearm terms were identified in 36 685 patient records (41.3%), 33.7% of snippets were categorized as definite access, 9.0% as definitely no access, and 57.2% as "other". Among models classifying firearm access, five of six had acceptable performance, with BioBERT and Bio-ClinicalBERT performing best, with F1s of 0.876 (95% confidence interval, 0.874-0.879) and 0.896 (95% confidence interval, 0.894-0.899), respectively.

Discussion and conclusion: Firearm-related terminology is common in the clinical records of VHA patients. The ability to use text to identify and characterize patients' firearm access could enhance suicide prevention efforts, and five of our six models could be used to identify patients for clinical interventions.

Objectives: This study aims to (1) review machine learning (ML)-based models for early infection diagnostic and prognosis prediction in post-acute care (PAC) settings, (2) identify key risk predictors influencing infection-related outcomes, and (3) examine the quality and limitations of these models.

Materials and methods: PubMed, Web of Science, Scopus, IEEE Xplore, CINAHL, and ACM digital library were searched in February 2024. Eligible studies leveraged PAC data to develop and evaluate ML models for infection-related risks. Data extraction followed the CHARMS checklist. Quality appraisal followed the PROBAST tool. Data synthesis was guided by the socio-ecological conceptual framework.

Results: Thirteen studies were included, mainly focusing on respiratory infections and nursing homes. Most used regression models with structured electronic health record data. Since 2020, there has been a shift toward advanced ML algorithms and multimodal data, biosensors, and clinical notes being significant sources of unstructured data. Despite these advances, there is insufficient evidence to support performance improvements over traditional models. Individual-level risk predictors, like impaired cognition, declined function, and tachycardia, were commonly used, while contextual-level predictors were barely utilized, consequently limiting model fairness. Major sources of bias included lack of external validation, inadequate model calibration, and insufficient consideration of data complexity.

Discussion and conclusion: Despite the growth of advanced modeling approaches in infection-related models in PAC settings, evidence supporting their superiority remains limited. Future research should leverage a socio-ecological lens for predictor selection and model construction, exploring optimal data modalities and ML model usage in PAC, while ensuring rigorous methodologies and fairness considerations.

Objective: This study aims to automate the prediction of Mini-Mental State Examination (MMSE) scores, a widely adopted standard for cognitive assessment in patients with Alzheimer's disease, using natural language processing (NLP) and machine learning (ML) on structured and unstructured EHR data.

Materials and methods: We extracted demographic data, diagnoses, medications, and unstructured clinical visit notes from the EHRs. We used Latent Dirichlet Allocation (LDA) for topic modeling and Term-Frequency Inverse Document Frequency (TF-IDF) for n-grams. In addition, we extracted meta-features such as age, ethnicity, and race. Model training and evaluation employed eXtreme Gradient Boosting (XGBoost), Stochastic Gradient Descent Regressor (SGDRegressor), and Multi-Layer Perceptron (MLP).

Results: We analyzed 1654 clinical visit notes collected between September 2019 and June 2023 for 1000 Alzheimer's disease patients. The average MMSE score was 20, with patients averaging 76.4 years old, 54.7% female, and 54.7% identifying as White. The best-performing model (ie, lowest root mean squared error (RMSE)) is MLP, which achieved an RMSE of 5.53 on the validation set using n-grams, indicating superior prediction performance over other models and feature sets. The RMSE on the test set was 5.85.

Discussion: This study developed a ML method to predict MMSE scores from unstructured clinical notes, demonstrating the feasibility of utilizing NLP to support cognitive assessment. Future work should focus on refining the model and evaluating its clinical relevance across diverse settings.

Conclusion: We contributed a model for automating MMSE estimation using EHR features, potentially transforming cognitive assessment for Alzheimer's patients and paving the way for more informed clinical decisions and cohort identification.

Objective: SNOMED CT provides a standardized terminology for clinical concepts, allowing cohort queries over heterogeneous clinical data including Electronic Health Records (EHRs). While it is intuitive that missing and inaccurate subtype (or is-a) relations in SNOMED CT reduce the recall and precision of cohort queries, the extent of these impacts has not been formally assessed. This study fills this gap by developing quantitative metrics to measure these impacts and performing statistical analysis on their significance.

Material and methods: We used the Optum de-identified COVID-19 Electronic Health Record dataset. We defined micro-averaged and macro-averaged recall and precision metrics to assess the impact of missing and inaccurate is-a relations on cohort queries. Both practical and simulated analyses were performed. Practical analyses involved 407 missing and 48 inaccurate is-a relations confirmed by domain experts, with statistical testing using Wilcoxon signed-rank tests. Simulated analyses used two random sets of 400 is-a relations to simulate missing and inaccurate is-a relations.

Results: Wilcoxon signed-rank tests from both practical and simulated analyses (P-values < .001) showed that missing is-a relations significantly reduced the micro- and macro-averaged recall, and inaccurate is-a relations significantly reduced the micro- and macro-averaged precision.

Discussion: The introduced impact metrics can assist SNOMED CT maintainers in prioritizing critical hierarchical defects for quality enhancement. These metrics are generally applicable for assessing the quality impact of a terminology's subtype hierarchy on its cohort query applications.

Conclusion: Our results indicate a significant impact of missing and inaccurate is-a relations in SNOMED CT on the recall and precision of cohort queries. Our work highlights the importance of high-quality terminology hierarchy for cohort queries over EHR data and provides valuable insights for prioritizing quality improvements of SNOMED CT's hierarchy.

Information visualizations are increasingly being developed by informatics researchers to communicate health information to lay audiences. For high-quality results, it is advisable to collaborate with creative professionals such as graphic designers, illustrators, or user interface/user experience designers. However, such collaborations are often a novel experience for both parties, each of which may be unfamiliar with the needs and processes of the other. We have coalesced our experiences from both the research and design perspectives to offer practical guidance in hopes of promoting the success of future collaborations. We offer suggestions for determining design needs, communicating with design professionals, and carrying out the design process. We assert that successful collaborations are predicated on careful and intentional planning at the outset of a project, a thorough understanding of each party's scope expertise, clear communication, and ample time for the design process to unfold.