AMIA Joint Summits on Translational Science proceedings. AMIA Joint Summits on Translational Science最新文献

Timing of clinical events is central to characterization of patient trajectories, enabling analyses such as process tracing, forecasting, and causal reasoning. However, structured electronic health records capture few data elements critical to these tasks, while clinical reports lack temporal localization of events in structured form. We present a system that transforms case reports into textual time series-structured pairs of textual events and timestamps. We contrast manual and large language model (LLM) annotations (n=320 and n=390 respectively) of ten randomly-sampled PubMed open-access (PMOA) case reports (N=152,974) and assess inter-LLM agreement (n=3,103 N=93). We find that the LLM models have moderate event recall (O1-preview: 0.80) but high temporal concordance among identified events (O1-preview: 0.95). By establishing the task, annotation, and assessment systems, and by demonstrating high concordance, this work may serve as a benchmark for leveraging the PMOA corpus for temporal analytics. Code is available at:https://github.com/jcweiss2/LLM-Timeline-PMOA/.

For evidence from observational studies to be reliable, researchers must ensure that the patient populations of interest are accurately defined. However, disease definitions can be extremely difficult to standardize and implement accurately across different datasets and study requirements. Furthermore, in this context, they must also ensure that populations are represented fairly to accurately reflect populations' various demographic dynamics and to not overgeneralize across non-applicable populations. In this work, we present a generalized tool to assess the fairness of disease definitions by evaluating their implementation across common fairness metrics. Our approach calculates fairness metrics and provides a robust method to examine coarse and strongly intersecting populations across many characteristics. We highlight workflows when working with disease definitions, provide an example analysis using an OMOP CDM patient database, and discuss potential directions for future improvement and research.

Low back pain (LBP) is a leading cause of disability globally. Following the onset of LBP and subsequent treatment, adequate patient education is crucial for improving functionality and long-term outcomes. Despite advancements in patient education strategies, significant gaps persist in delivering personalized, evidence-based information to patients with LBP. Recent advancements in large language models (LLMs) and generative artificial intelligence (GenAI) have demonstrated the potential to enhance patient education. However, their application and efficacy in delivering educational content to patients with LBP remain underexplored and warrant further investigation. In this study, we introduce a novel approach utilizing LLMs with Retrieval-Augmented Generation (RAG) and few-shot learning to generate tailored educational materials for patients with LBP. Physical therapists manually evaluated our model responses for redundancy, accuracy, and completeness using a Likert scale. In addition, the readability of the generated education materials is assessed using the Flesch Reading Ease score. The findings demonstrate that RAG-based LLMs outperform traditional LLMs, providing more accurate, complete, and readable patient education materials with less redundancy. Having said that, our analysis reveals that the generated materials are not yet ready for use in clinical practice. This study underscores the potential of AI-driven models utilizing RAG to improve patient education for LBP; however, significant challenges remain in ensuring the clinical relevance and granularity of content generated by these models.

Social determinants of health (SDoH) play a crucial role in patient health outcomes, yet their integration into biomedical knowledge graphs remains underexplored. This study addresses this gap by constructing an SDoH-enriched knowledge graph using the MIMIC-III dataset and PrimeKG. We introduce a novel fairness formulation for graph embeddings, focusing on invariance with respect to sensitive SDoH information. Via employing a heterogeneous-GCN model for drug-disease link prediction, we detect biases related to various SDoH factors. To mitigate these biases, we propose a post-processing method that strategically reweights edges connected to SDoHs, balancing their influence on graph representations. This approach represents one of the first comprehensive investigations into fairness issues within biomedical knowledge graphs incorporating SDoH. Our work not only highlights the importance of considering SDoH in medical informatics but also provides a concrete method for reducing SDoH-related biases in link prediction tasks, paving the way for more equitable healthcare recommendations. Our code is available at https://github.com/hwq0726/SDoH-KG.

Youth who experience concussions may be at greater risk for subsequent mental health challenges, making early detection crucial for timely intervention. This study utilized Bidirectional Long Short-Term Memory (BiLSTM) networks to predict mental health outcomes following concussion in youth and compared its performance to traditional models. We also examined whether incorporating social determinants of health (SDoH) improved predictive power, given the disproportionate impact of concussions and mental health issues on disadvantaged populations. We evaluated the models using accuracy, area under the curve (4UC) of the receiver operating characteristic (ROC), and other performance metrics. Our BiLSTM model with SDoH data achieved the highest accuracy (0.883) and 4UC-ROC score (0.892). Unlike traditional models, our approach provided real-time predictions at each visit within 12 months of the index concussion, aiding clinicians in making timely, visit-specific referrals for further treatment and interventions.

Machine Learning (ML) algorithms are vital for supporting clinical decision-making in biomedical informatics. However, their predictive performance can vary across demographic groups, often due to the underrepresentation of historically marginalized populations in training datasets. The investigation reveals widespread sex- and age-related inequities in chronic disease datasets and their derived ML models. Thus, a novel analytical framework is introduced, combining systematic arbitrariness with traditional metrics like accuracy and data complexity. The analysis of data from over 25,000 individuals with chronic diseases revealed mild sex-related disparities, favoring predictive accuracy for males, and significant age-related differences, with better accuracy for younger patients. Notably, older patients showed inconsistent predictive accuracy across seven datasets, linked to higher data complexity and lower model performance. This highlights that representativeness in training data alone does not guarantee equitable outcomes, and model arbitrariness must be addressed before deploying models in clinical settings.

Wearable devices collect complex structured data with high-dimensional and time-series features that are challenging for traditional models to handle efficiently. We propose EntroLLM, a new method that combines entropy measures and the low-dimensional representation (embedding) generated from large language models (LLMs) to enhance risk prediction using wearable device data. In EntroLLM, the entropy quantifies the variability of a subject's physical activity patterns, while the LLM embedding approximates the latent temporal structure. We evaluate the feasibility and performance of EntroLLM using NHANES data to predict overweight status using demographics and physical activity collected from wearable devices. Results show that combining entropy with GPT-based embedding improves model performance compared to baseline models and other embedding techniques, leading to an average increase in AUC from 0.56 to 0.64. EntroLLM showcases the potential of combining entropy and LLM-based embedding and offers a promising approach to wearable device data analysis for predicting health outcomes.

Heart failure (HF) is a significant public health challenge, especially among critically ill patients in intensive care units (ICUs). Predicting survival outcomes for these patients with calibrated uncertainty is both challenging and essential for guiding subsequent treatments. This study introduces conformalized survival analysis (CSA) as a novel method for predicting survival times in critically ill HF patients. CSA enhances each predicted survival time with a statistically rigorous lower bound, providing valuable uncertainty quantification. Using the MIMIC-IV dataset, we demonstrate that CSA effectively delivers calibrated uncertainty quantification for survival predictions, in contrast to parametric models like the Cox or Accelerated Failure Time models. Through the application of CSA to a large, real-world dataset, this study underscores its potential to improve decision-making in critical care, offering a more precise and reliable tool for prognosis in a setting where accurate predictions and calibrated uncertainty can profoundly impact patient outcomes.

Personalized medicine aims to improve clinical outcomes by tailoring treatments to individual patients based on genetic, phenotypic, or psychosocial characteristics, leveraging insights from similar patients. This is particularly necessary for managing diseases with significant variability in their causes, progressions and prognoses. Accurate measurement of patient similarity is crucial in this context, as it enables the identification of a high-quality cohort of similar patients, thereby enhancing clinical decision making with better evidence. However, previous studies have not comprehensively compared different patient similarity measures in large-scale retrospective analyses of electronic health records (EHRs). In this study, we conducted a comparative analysis of four patient similarity measures focusing on feature weighting mechanisms, using EHR data from 46,968 hospitalized patients. For evaluation, we assessed the patient similarity measures for predicting acute kidney injury, readmission, and mortality. Our results showed that using grid-searched weights to combine features based by their types outperformed all other methods.

Several use cases in precision oncology require accurately extracting and standardizing Real-World Data from Electronic Health Records (EHRs). We developed the infrastructure and a toolset incorporating data mining and natural language processing scripts to automatically retrieve selected descriptive and common endpoint variables from EHRs. This toolset was evaluated against a reference dataset of 106 lung cancer and 45 sarcoma patient cases pulled from two databases complying with the Precision Oncology Core Data Model (Precision-DM) and maintained by the Johns Hopkins Molecular Tumor Board and a research team. We accurately retrieved most descriptive EHR fields but less efficiently extracted the Date of Diagnosis and Treatment Start Date that supported calculating the Age at Diagnosis, Overall Survival, and Time to First Treatment (accuracy range 50%-86%). Our infrastructure and Precision-DM-based standardization could inspire similar efforts in other cancer centers, however, the toolset should be enhanced to improve accuracy in certain variables.