AMIA ... Annual Symposium proceedings. AMIA Symposium最新文献

Large Language Models (LLMs), although powerful in general domains, often perform poorly on domain-specific tasks such as medical question answering (QA). In addition, LLMs tend to function as "black-boxes", making it challenging to modify their behavior. To address the problem, our work employs a transparent process of retrieval augmented generation (RAG), aiming to improve LLM responses without the need for fine-tuning or retraining. Specifically, we propose a comprehensive retrieval strategy to extract medical facts from an external knowledge base, and then inject them into the LLM's query prompt. Focusing on medical QA, we evaluate the impact of different retrieval models and the number of facts on LLM performance using the MedQA-SMILE dataset. Notably, our retrieval-augmented Vicuna-7B model exhibited an accuracy improvement from 44.46% to 48.54%. This work underscores the potential of RAG to enhance LLM performance, offering a practical approach to mitigate the challenges posed by black-box LLMs.

Medication persistence is essential for the efficacy of treatment and patient health outcomes. This study investigates the discontinuation of oral anticancer medications (capecitabine, ibrutinib, or sunitinib) in a cohort that is well-characterized by medication discontinuation survey questionnaires, prescription refill data, and structured electronic health records (EHRs). We categorized discontinuation reasons based on a survey of patients taking medication, revealing that 38% of 257 patients completed therapy, while discontinuation was due primarily to no response to therapy and/or progression of disease leading to discontinuation (33%) and side effects/complication (9%). Survival analysis showed variable medication persistence, with capecitabine persistence decreasing significantly over time, to 0.08 in two years. A logistic regression model demonstrated strong capability (with an AUC of 0.835) to identify patients at risk for medication discontinuation. Our study shows the complexities of medication persistence and emphasizes the importance of understanding medication discontinuation patterns and leveraging predictive analytics to inform future research and clinical monitoring in the treatment of cancer.

Electronic Health Records (EHRs) are valuable healthcare data, aiding researchers and doctors in improving diagnosis accuracy. Researchers have developed several predictive models by learning disease representations to forecast the potential diagnosis that patients may receive. However, existing studies usually ignore the fine-grained semantic and structure information in EHRs (e.g., the hierarchical relations between diseases and ICD-9 codes), which fails to provide accurate disease representation towards effective diagnosis prediction. To this end, we propose to enhance diagnosis prediction through LabCare, a framework with improved semantic and structure modeling of diseases in EHR data. LabCare can simultaneously capture rich semantic and structural relations among diseases and ICD-9 codes, which is achieved by innovatively integrating language models and box embeddings. Extensive experiments on two EHR datasets show that LabCare surpasses competitors, consistently achieving a 4.29% average improvement in Recall and NDCG metrics.

This capstone project investigates the application of artificial intelligence (AI) techniques, specifically sentence embedding and k-means clustering using large language models, to address the challenge of policy library redundancy within a healthcare setting. The project aimed to demonstrate the viability of using AI-assisted tools in policy library management, targeting a 5% reduction in the overall policy library at a large academic healthcare system. By collaborating with the accreditation team and developing a Python-script prototype, the study showed that AI-assisted methods could significantly enhance efficiency and reduce labor in policy library management. Results indicate a potential 4% reduction in library size, underscoring the method's effectiveness and the opportunity for further optimization. This research contributes to the emerging field of AI in healthcare administration, offering a scalable model for improving policy library management processes in various healthcare contexts.

Successful integration of Generative Artificial Intelligence (AI) into healthcare requires understanding of health professionals' perspectives, ideally through data-driven approaches. In this study, we use a semi-structured survey and mixed methods analyses to explore clinicians' perceptions on the utility of generative AI for all types of clinical tasks, familiarity and competency with generative AI tools, and sentiments regarding the potential impact of generative AI on healthcare. Analysis of 116 clinician responses found differing perceptions regarding the usefulness of generative AI across clinical workflows, with information gathering from external sources rated highest and communication rated lowest. Clinician-generated prompt suggestions focused most often on clinician decision making and were of mixed quality, with participants more familiar with generative AI suggesting more high-quality prompts. Sentiments regarding the impact of generative AI varied, particularly regarding trustworthiness and impact on bias. Thematic analysis of open-ended comments highlighted concerns about patient care and the role of clinicians.

Evidence based medicine and health data for policy should update statistical data modeling methods to take advantage of at-scale data. One challenge with at-scale data is information segmentation for clinical presentation discovery and cohort assignment. We use gradient boosting machine (GBM) to segment 94,379,175,015 diagnostic clinical events attributable to 283,632,789 Centers for Medicare and Medicaid Services beneficiaries over 22 observation years. Diagnostic events were aggregated into attack rates by demography and Phenome-wide association studies (PheWas) codes. Resulting attack rates were then segmented within a user defined clinical status of interest, in this case HIV status. 1,753,647 HIV+ beneficiaries were considered. The GBM model assigned 19,651,408 PheWas attack rates from 69,133,296 ICD attack rates into phenogroups/nodes.

Large language models (LLMs) have shown great promise in clinical medicine, but their adoption in real-world settings has been limited by their tendency to generate incorrect and sometimes even toxic statements. This study presents a reliable rare disease intelligent agent called RDguru, which incorporates authoritative and reliable knowledge sources and tools into the reasoning and response of LLMs. In addition to answering questions about rare diseases more accurately, RDguru can conduct medical consultations to provide differential diagnosis decision support for clinical users. The DQN-based multi-source fusion diagnostic model integrates three diagnostic recommendation strategies, GPT-4, PheLR, and phenotype matching. Testing on 238 real rare disease cases showed that RDguru's top 10 list of recommended diagnoses was able to recall 69.1% of real diagnoses, the top 5 recommended diagnoses were able to recall 63.6% of real diagnoses, and the top ranked diagnosis was able to achieve an accuracy rate of 41.9%.

The Centers for Disease Control and Prevention has raised national alarm over five Antimicrobial Resistant Organisms (AMROs) considered urgent or serious threats to public safety. Understanding the prevalence and distribution of AMROs at a local level can inform the unique infection risks facing our communities. We conducted a retrospective, spatiotemporal analysis of AMRO prevalence across Tarrant County, Texas from 2010-2019. Using spatial autocorrelation tests, we identified that across five different AMRO subtypes, the Western half of Tarrant County experienced more hot spots than the Eastern half. Our Space-Time Permutation Models identified 35 unique AMRO clusters. Using logistic regression models, we found significant associations between Area Deprivation Index, a measure of socioeconomic disparity, and most AMRO clusters. These findings underscore the importance of residency location and temporal trends when treating and preventing AMRO infections.

Excessive documentation burden is linked to clinician burnout, thus motivating efforts to reduce burden. Generative artificial intelligence (AI) poses opportunities for burden reduction but requires rigorous assessment. We evaluated the ability of a large language model (LLM) (OpenAI's GPT-4) to interpret various intervention-response relationships presented on nursing flowsheets, assessing performance using MUC-5 evaluation metrics, and compared its assessments to those of nurse expert evaluators. ChatGPT correctly assessed 3 of 14 clinical scenarios, and partially correctly assessed 6 of 14, frequently omitting data from its reasoning. Nurse expert evaluators correctly assessed all relationships and provided additional language reflective of standard nursing practice beyond the intervention-response relationships evidenced in nursing flowsheets. Future work should ensure the training data used for electronic health record (EHR)-integrated LLMs includes all types of narrative nursing documentation that reflect nurses' clinical reasoning, and verification of LLM-based information summarization does not burden end-users.

Pre-hospital communication, which usually refers to the communication process between pre-hospital and hospital providers, is crucial for the effective management of critically injured or ill patients. Despite its importance, persistent challenges such as miscommunication have been significant barriers. Telemedicine systems have been proposed to overcome these challenges, yet existing research primarily focuses on using off-the-shelf systems to evaluate their feasibility and effectiveness of implementation without investigating users' needs and perceptions. To bridge this research gap, our study employed a user-centered design approach to co-create an integrated telemedicine system with emergency care providers to ensure that the system meets the specific needs of care providers and aligns with existing clinical workflows. We present the system design process, the features desired by users to address challenges in pre-hospital communication, and the socio-technical considerations for implementing telemedicine in the dynamic emergency care setting. We conclude the paper by discussing the design implications.