Xueting Wang, Lucila Ohno-Machado, Jose L Gomez, Wen Gu, Rongyi Sun, Jihoon Kim
While there is a growing recognition of the association between depression and asthma, few studies have leveraged deep learning-based (DL-based) models in a retrospective cohort study with a large sample size. We analyzed the association between depression and asthma among 239,161 participants of the All of Us Research Program through DL-based, logistic regression, and Cox Proportional Hazards (CoxPH) models. We used SHAP values to help interpret DL-based models and c-index to evaluate model performance. Results suggest a significant odds ratio for depression in asthma. The c-indices for the CoxPH, DeepSurv, and DeepHit models were 0.619, 0.625, and 0.596, respectively. SHAP indicated a different set of important variables when compared with the CoxPH model. In conclusion, we provide strong evidence of a positive relationship between depression and asthma. Also, DL-based models did not outperform the CoxPH model on the c-index. Sex at birth and income may play important roles in occurrence of depression in asthma patients.
虽然人们越来越认识到抑郁症和哮喘之间的联系,但很少有研究在大样本量的回顾性队列研究中利用基于深度学习(基于dl)的模型。我们通过基于dl的logistic回归和Cox比例风险(Cox Proportional Hazards, Cox)模型分析了239161名All of Us研究项目参与者的抑郁和哮喘之间的关系。我们使用SHAP值来帮助解释基于dl的模型,并使用c-index来评估模型的性能。结果显示哮喘患者抑郁的优势比显著。CoxPH、DeepSurv和DeepHit模型的c指数分别为0.619、0.625和0.596。与CoxPH模型相比,SHAP表明了一组不同的重要变量。总之,我们提供了强有力的证据证明抑郁和哮喘之间存在正相关关系。此外,基于dl的模型在c指数上也没有优于CoxPH模型。出生性别和收入可能在哮喘患者抑郁的发生中起重要作用。
{"title":"Deep Learning-based Time-to-event Analysis of Depression and Asthma using the All of Us Research Program.","authors":"Xueting Wang, Lucila Ohno-Machado, Jose L Gomez, Wen Gu, Rongyi Sun, Jihoon Kim","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>While there is a growing recognition of the association between depression and asthma, few studies have leveraged deep learning-based (DL-based) models in a retrospective cohort study with a large sample size. We analyzed the association between depression and asthma among 239,161 participants of the All of Us Research Program through DL-based, logistic regression, and Cox Proportional Hazards (CoxPH) models. We used SHAP values to help interpret DL-based models and c-index to evaluate model performance. Results suggest a significant odds ratio for depression in asthma. The c-indices for the CoxPH, DeepSurv, and DeepHit models were 0.619, 0.625, and 0.596, respectively. SHAP indicated a different set of important variables when compared with the CoxPH model. In conclusion, we provide strong evidence of a positive relationship between depression and asthma. Also, DL-based models did not outperform the CoxPH model on the c-index. Sex at birth and income may play important roles in occurrence of depression in asthma patients.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"1186-1195"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12099346/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144144444","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The advent of clinical language models integrated into electronic health records (EHR) for clinical decision support has marked a significant advancement, leveraging the depth of clinical notes for improved decision-making. Despite their success, the potential vulnerabilities of these models remain largely unexplored. This paper delves into the realm of backdoor attacks on clinical language models, introducing an innovative attention-based backdoor attack method, BadCLM (Bad Clinical Language Models). This technique clandestinely embeds a backdoor within the models, causing them to produce incorrect predictions when a pre-defined trigger is present in inputs, while functioning accurately otherwise. We demonstrate the efficacy of BadCLM through an in-hospital mortality prediction task with MIMIC III dataset, showcasing its potential to compromise model integrity. Our findings illuminate a significant security risk in clinical decision support systems and pave the way for future endeavors in fortifying clinical language models against such vulnerabilities.
集成到用于临床决策支持的电子健康记录(EHR)中的临床语言模型的出现标志着一个重大进步,它利用临床记录的深度来改进决策。尽管它们取得了成功,但这些模型的潜在漏洞在很大程度上仍未被探索。本文深入研究了临床语言模型的后门攻击领域,引入了一种创新的基于注意力的后门攻击方法BadCLM (Bad clinical language models)。这种技术秘密地在模型中嵌入了一个后门,导致它们在输入中存在预定义触发器时产生不正确的预测,而在其他情况下则准确运行。我们通过使用MIMIC III数据集的住院死亡率预测任务证明了BadCLM的有效性,展示了其损害模型完整性的潜力。我们的研究结果阐明了临床决策支持系统中存在的重大安全风险,并为未来加强临床语言模型以应对此类漏洞铺平了道路。
{"title":"BadCLM: Backdoor Attack in Clinical Language Models for Electronic Health Records.","authors":"Weimin Lyu, Zexin Bi, Fusheng Wang, Chao Chen","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>The advent of clinical language models integrated into electronic health records (EHR) for clinical decision support has marked a significant advancement, leveraging the depth of clinical notes for improved decision-making. Despite their success, the potential vulnerabilities of these models remain largely unexplored. This paper delves into the realm of backdoor attacks on clinical language models, introducing an innovative attention-based backdoor attack method, BadCLM (Bad Clinical Language Models). This technique clandestinely embeds a backdoor within the models, causing them to produce incorrect predictions when a pre-defined trigger is present in inputs, while functioning accurately otherwise. We demonstrate the efficacy of BadCLM through an in-hospital mortality prediction task with MIMIC III dataset, showcasing its potential to compromise model integrity. Our findings illuminate a significant security risk in clinical decision support systems and pave the way for future endeavors in fortifying clinical language models against such vulnerabilities.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"768-777"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12099347/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144144701","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Subiksha Umakanth, Anna Vaynrub, Harry West, Jill Dimond, Alissa Michel, Katherine D Crew, Rita Kukafka
RealRisks is a decision aid that integrates patient-generated and electronic health record (EHR) data using Fast Healthcare Interoperability Resources (FHIR). It offers modules to enhance understanding of breast cancer risk and a way for individuals to review and modify their EHR data before it is used in their personal risk assessment. RealRisks intends to encourage high-risk patients to take risk-reducing measures. To better understand how patients understand risk and barriers to action, we conducted in-depth interviews as part of a usability study to assess the clarity and interpretability of RealRisks. Overall, participants demonstrated an improved understanding of breast cancer risk after using RealRisks. However, challenges were noted for certain concepts, in particular, lifetime risk, how benign breast disease affects your risk, and the differences between hereditary, sporadic, and familial cancer. The EHR download feature was well-received, but some raised concerns about insurance and privacy/security.
{"title":"User Comprehension and EHR Integration of the <i>RealRisks</i> Decision Aid for Breast Cancer Risk Assessment: A Qualitative Study.","authors":"Subiksha Umakanth, Anna Vaynrub, Harry West, Jill Dimond, Alissa Michel, Katherine D Crew, Rita Kukafka","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>RealRisks is a decision aid that integrates patient-generated and electronic health record (EHR) data using Fast Healthcare Interoperability Resources (FHIR). It offers modules to enhance understanding of breast cancer risk and a way for individuals to review and modify their EHR data before it is used in their personal risk assessment. RealRisks intends to encourage high-risk patients to take risk-reducing measures. To better understand how patients understand risk and barriers to action, we conducted in-depth interviews as part of a usability study to assess the clarity and interpretability of RealRisks. Overall, participants demonstrated an improved understanding of breast cancer risk after using RealRisks. However, challenges were noted for certain concepts, in particular, lifetime risk, how benign breast disease affects your risk, and the differences between hereditary, sporadic, and familial cancer. The EHR download feature was well-received, but some raised concerns about insurance and privacy/security.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"1129-1138"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12099428/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"144144859","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The All of Us (AoU) Research Program and UK Biobank (UKBB) boast a wealth of EHR data, which can be harnessed to refine cohort selection via rule-based phenotyping algorithms. The Observational Health Data Sciences and Informatics (OHDSI) Phenotype Library (PL) hosts many complex phenotyping rules. Here, we compare prevalence for 423 OHDSI PL cohorts in AoU and UKBB. For three select diseases (T2D, COPD, Acute MI), we analyze differences in demographics, social determinants of health (SDOH), geographic prevalence, and genome-wide association study (GWAS) results. We found that AoU has a significantly higher prevalence for 80% of phenotypes compared to UKBB. We also found that for the select diseases, SDOH variables between the two biobanks differ significantly. Findings for each of these three diseases confirm known regions of high risk. Additionally, GWAS in UKBB discovered more genes associated with each of the three diseases than GWAS in AoU.
{"title":"Cross Biobank Comparison of Phenomic Profiles.","authors":"Abigail Newbury, Xinzhuo Jiang, Karthik Natarajan, Gamze Gürsoy","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>The <i>All of Us</i> (AoU) Research Program and UK Biobank (UKBB) boast a wealth of EHR data, which can be harnessed to refine cohort selection via rule-based phenotyping algorithms. The Observational Health Data Sciences and Informatics (OHDSI) Phenotype Library (PL) hosts many complex phenotyping rules. Here, we compare prevalence for 423 OHDSI PL cohorts in AoU and UKBB. For three select diseases (T2D, COPD, Acute MI), we analyze differences in demographics, social determinants of health (SDOH), geographic prevalence, and genome-wide association study (GWAS) results. We found that AoU has a significantly higher prevalence for 80% of phenotypes compared to UKBB. We also found that for the select diseases, SDOH variables between the two biobanks differ significantly. Findings for each of these three diseases confirm known regions of high risk. Additionally, GWAS in UKBB discovered more genes associated with each of the three diseases than GWAS in AoU.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"939-948"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919452/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147272640","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Chanhee Kim, Aasa Dahlberg Schmit, Sarah Solarz, Sripriya Rajamani
With data considered as the 'oxygen' of public health, the Data Modernization Initiative (DMI) to enhance the public health data and information infrastructure is critical. The DMI Stories from the Field features data modernization from public health agencies to highlight success/progress. These stories (n=241) were analyzed, with outbreak response, information systems capacity, epidemiology/laboratory capacity being some of the common topics. A total of 199 codes across DMI stories were organized into 7 themes and the top 3 codes were communication, collaboration and public health agencies. Key takeaways and next steps were identified and validated with expert input across people, product, process and partnership categories and people factor was critical along with funding/sustainability. Ongoing DMI stories and future studies for evaluating impact are recommended. DMI stories are a great option to communicate the projects and impact of DMI to a larger public audience and garner support for this vital endeavor.
{"title":"Data Modernization in Action: Synthesizing Pioneering Informatics Projects in Public Health and Data Modernization Stories from Public Health Agencies.","authors":"Chanhee Kim, Aasa Dahlberg Schmit, Sarah Solarz, Sripriya Rajamani","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>With data considered as the 'oxygen' of public health, the Data Modernization Initiative (DMI) to enhance the public health data and information infrastructure is critical. The DMI Stories from the Field features data modernization from public health agencies to highlight success/progress. These stories (n=241) were analyzed, with outbreak response, information systems capacity, epidemiology/laboratory capacity being some of the common topics. A total of 199 codes across DMI stories were organized into 7 themes and the top 3 codes were communication, collaboration and public health agencies. Key takeaways and next steps were identified and validated with expert input across people, product, process and partnership categories and people factor was critical along with funding/sustainability. Ongoing DMI stories and future studies for evaluating impact are recommended. DMI stories are a great option to communicate the projects and impact of DMI to a larger public audience and garner support for this vital endeavor.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"605-614"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919559/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147272663","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Large Language Model (LLM)-based conversational agents offer promising solutions for mental health support, but lack cultural responsiveness for diverse populations. This study evaluated the effectiveness of cultural prompting in improving cultural responsiveness and perceived empathy of LLM-generated therapeutic responses for Chinese American family caregivers. Using a randomized controlled experiment, we compared GPT-4o and Deepseek-V3 responses with and without cultural prompting. Thirty-six participants evaluated input-response pairs on cultural responsiveness (competence and relevance) and perceived empathy. Results showed that cultural prompting significantly enhanced GPT-4o's performance across all dimensions, with GPT-4o with cultural prompting being the most preferred, while improvements in DeepSeek-V3 responses were not significant. Mediation analysis revealed that cultural prompting improved empathy through improving cultural responsiveness. This study demonstrated that prompt-based techniques can effectively enhance the cultural responsiveness of LLM-generated therapeutic responses, highlighting the importance of cultural responsiveness in delivering empathetic AI-based therapeutic interventions to culturally and linguistically diverse populations.
{"title":"Cultural Prompting Improves the Empathy and Cultural Responsiveness of GPT-Generated Therapy Responses.","authors":"Serena Jinchen Xie, Shumenghui Zhai, Yanjing Liang, Jingyi Li, Xuehong Fan, Trevor Cohen, Weichao Yuwen","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Large Language Model (LLM)-based conversational agents offer promising solutions for mental health support, but lack cultural responsiveness for diverse populations. This study evaluated the effectiveness of cultural prompting in improving cultural responsiveness and perceived empathy of LLM-generated therapeutic responses for Chinese American family caregivers. Using a randomized controlled experiment, we compared GPT-4o and Deepseek-V3 responses with and without cultural prompting. Thirty-six participants evaluated input-response pairs on cultural responsiveness (competence and relevance) and perceived empathy. Results showed that cultural prompting significantly enhanced GPT-4o's performance across all dimensions, with GPT-4o with cultural prompting being the most preferred, while improvements in DeepSeek-V3 responses were not significant. Mediation analysis revealed that cultural prompting improved empathy through improving cultural responsiveness. This study demonstrated that prompt-based techniques can effectively enhance the cultural responsiveness of LLM-generated therapeutic responses, highlighting the importance of cultural responsiveness in delivering empathetic AI-based therapeutic interventions to culturally and linguistically diverse populations.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"1384-1393"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919561/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147272674","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Terika McCall, Amelea Lowery, Bria Massey, Meera Swaminath, Shamima Afrose, Monya Saunders, Karen H Wang
This study explores the challenges faced by justice-impacted Black women during their reintegration into society, with a focus on mental health care access and the potential for technology-assisted interventions to address barriers. Participants from focus groups emphasized significant obstacles, including inadequate mental health resources during incarceration, insufficient post-release support, and barriers such as discrimination, lack of insurance, and transportation issues. When designing technology-assisted interventions, such as the Welcome Home app, additional considerations for justice-impacted Black women include trauma-informed design, tiered support systems, integration with electronic health records, privacy protection, and culturally tailored content. The study underscores the importance of culturally relevant, user-centered digital solutions to improve health outcomes and facilitate the successful reintegration of Black women impacted by the criminal legal system. Apps that provide a sense of community promote engagement, which may improve health outcomes.
{"title":"Designing Technology-Assisted Interventions for Justice-Impacted Black American Women.","authors":"Terika McCall, Amelea Lowery, Bria Massey, Meera Swaminath, Shamima Afrose, Monya Saunders, Karen H Wang","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>This study explores the challenges faced by justice-impacted Black women during their reintegration into society, with a focus on mental health care access and the potential for technology-assisted interventions to address barriers. Participants from focus groups emphasized significant obstacles, including inadequate mental health resources during incarceration, insufficient post-release support, and barriers such as discrimination, lack of insurance, and transportation issues. When designing technology-assisted interventions, such as the Welcome Home app, additional considerations for justice-impacted Black women include trauma-informed design, tiered support systems, integration with electronic health records, privacy protection, and culturally tailored content. The study underscores the importance of culturally relevant, user-centered digital solutions to improve health outcomes and facilitate the successful reintegration of Black women impacted by the criminal legal system. Apps that provide a sense of community promote engagement, which may improve health outcomes.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"854-860"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919450/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147272884","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Lauren N Cooper, Aamirah Vadsariya, Mereeja Varghese, Bhavini Nayee, Jessica Moon, Chaitanya Katterapalli, Clark Walker, Chris Gonzalez, Sonam Sohal, Christoph U Lehmann, Ferdinand Velasco, Mujeeb Basit, DuWayne Willett
The University of Texas Southwestern Medical Center (UTSW) and Texas Health Resources (THR) implemented an Observational Medical Outcomes Partnership (OMOP) common data model (CDM) that utilizes the Epic electronic health record's (EHR) extract, transform, and load (ETL) system to enable collaborative research with other health institutions within the OHDSI network. We mapped EHR data from core Epic reporting tables to 25 OMOP CDM tables and transferred the data to a shared OMOP database housed within the Caboodle infrastructure using Epic's pre-existing ETL system, minimizing the need for customization. ETL processes occur weekly at THR and daily at UTSW. OMOP CDM mapping resulted in data quality assessment values of 97% and 98% for THR and UTSW respectively. Our study established a reproduceable, collaborative pipeline using the OMOP CDM with Epic's native ETL framework, expanding the OHDSI research network resulting in better quality and more generalizable data sets available for future research.
{"title":"LEVERAGING EPIC'S NATIVE ETL INFRASTRUCTURE FOR OMOP CDM IMPLEMENTATION: A COLLABORATIVE EXPERIENCE.","authors":"Lauren N Cooper, Aamirah Vadsariya, Mereeja Varghese, Bhavini Nayee, Jessica Moon, Chaitanya Katterapalli, Clark Walker, Chris Gonzalez, Sonam Sohal, Christoph U Lehmann, Ferdinand Velasco, Mujeeb Basit, DuWayne Willett","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>The University of Texas Southwestern Medical Center (UTSW) and Texas Health Resources (THR) implemented an Observational Medical Outcomes Partnership (OMOP) common data model (CDM) that utilizes the Epic electronic health record's (EHR) extract, transform, and load (ETL) system to enable collaborative research with other health institutions within the OHDSI network. We mapped EHR data from core Epic reporting tables to 25 OMOP CDM tables and transferred the data to a shared OMOP database housed within the Caboodle infrastructure using Epic's pre-existing ETL system, minimizing the need for customization. ETL processes occur weekly at THR and daily at UTSW. OMOP CDM mapping resulted in data quality assessment values of 97% and 98% for THR and UTSW respectively. Our study established a reproduceable, collaborative pipeline using the OMOP CDM with Epic's native ETL framework, expanding the OHDSI research network resulting in better quality and more generalizable data sets available for future research.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"287-292"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919598/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147272960","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Meagan Foster, Elizabeth Byrd, Elizabeth Kwong, Anirudh Karunaker, Brian M Anderson, Michael C Repka, Ross McGurk, Shiva K Das, Lawrence B Marks, Lukasz Mazur
Clinical variability in prostate radiation therapy (RT) planning is well documented, but little is known about how radiation oncologists experience and adapt to the factors that drive it. This study explores variability as a human-centered design challenge, with the goalofinformingclinicaldecision support (CDS) design through real-timeinsight into planning decisions. We conducted observation sessions with the think aloud method followed by semi-structured interviews with five radiation oncologists while they contoured prostate cases. Using the Systems Engineering Initiative for Patient Safety (SEIPS) framework, we thematically analyzed the contributors to variability across tasks, technology, and organizational conditions. Results suggest that variability arises not only from anatomical or guidelineambiguity, butalso fromindividual interpretations of inputs, variation in contouring decisions, andadaptive strategies such as reliance on prior experience and estimation under uncertainty. Findings support the design of context-sensitive CDS tools that reflect real-world clinical reasoning while preserving clinical flexibility.
{"title":"Navigating Variability in Prostate RT Planning: Real-Time Insights for Human-Centered CDS Design.","authors":"Meagan Foster, Elizabeth Byrd, Elizabeth Kwong, Anirudh Karunaker, Brian M Anderson, Michael C Repka, Ross McGurk, Shiva K Das, Lawrence B Marks, Lukasz Mazur","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Clinical variability in prostate radiation therapy (RT) planning is well documented, but little is known about how radiation oncologists experience and adapt to the factors that drive it. This study explores variability as a human-centered design challenge, with the goalofinformingclinicaldecision support (CDS) design through real-timeinsight into planning decisions. We conducted observation sessions with the think aloud method followed by semi-structured interviews with five radiation oncologists while they contoured prostate cases. Using the Systems Engineering Initiative for Patient Safety (SEIPS) framework, we thematically analyzed the contributors to variability across tasks, technology, and organizational conditions. Results suggest that variability arises not only from anatomical or guidelineambiguity, butalso fromindividual interpretations of inputs, variation in contouring decisions, andadaptive strategies such as reliance on prior experience and estimation under uncertainty. Findings support the design of context-sensitive CDS tools that reflect real-world clinical reasoning while preserving clinical flexibility.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"369-375"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919530/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273002","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dongfang Xu, Guillermo López García, Karen O'Connor, Haily Holston, Ari Z Klein, Ivan Flores Amaro, Matthew Scotch, Graciela Gonzalez-Hernandez
Influenza vaccine effectiveness (VE) estimation plays a critical role in public health decision-making by quantifying the real-world impact of vaccination campaigns and guiding policy adjustments. Current approaches to VE estimation are constrained by limited population representation, selection bias, and delayed reporting. To address some of these gaps, we propose leveraging large language models (LLMs) with few-shot chain-of-thought (CoT) prompting to mine social media data for real-time influenza VE estimation. We annotated over 4,000 tweets from the 2020-2021 flu season using structured guidelines, achieving high inter-annotator agreement. Our best prompting strategy achieves F1 scores above 87% for identifying influenza vaccination status and test outcomes, outperforming traditional supervised fine-tuning methods by large margins. These findings indicate that LLM-based prompting approaches effectively identify relevant social media information for influenza VE estimation, offering a valuable real-time surveillance tool that complements traditional epidemiological methods.
{"title":"Mining Social Media Data for Influenza Vaccine Effectiveness Using a Large Language Model and Chain-of-Thought Prompting.","authors":"Dongfang Xu, Guillermo López García, Karen O'Connor, Haily Holston, Ari Z Klein, Ivan Flores Amaro, Matthew Scotch, Graciela Gonzalez-Hernandez","doi":"","DOIUrl":"","url":null,"abstract":"<p><p>Influenza vaccine effectiveness (VE) estimation plays a critical role in public health decision-making by quantifying the real-world impact of vaccination campaigns and guiding policy adjustments. Current approaches to VE estimation are constrained by limited population representation, selection bias, and delayed reporting. To address some of these gaps, we propose leveraging large language models (LLMs) with few-shot chain-of-thought (CoT) prompting to mine social media data for real-time influenza VE estimation. We annotated over 4,000 tweets from the 2020-2021 flu season using structured guidelines, achieving high inter-annotator agreement. Our best prompting strategy achieves F<sub>1</sub> scores above 87% for identifying influenza vaccination status and test outcomes, outperforming traditional supervised fine-tuning methods by large margins. These findings indicate that LLM-based prompting approaches effectively identify relevant social media information for influenza VE estimation, offering a valuable real-time surveillance tool that complements traditional epidemiological methods.</p>","PeriodicalId":72180,"journal":{"name":"AMIA ... Annual Symposium proceedings. AMIA Symposium","volume":"2024 ","pages":"1404-1413"},"PeriodicalIF":0.0,"publicationDate":"2025-05-22","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC12919567/pdf/","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"147273038","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}