Pub Date : 2024-07-21DOI: 10.1101/2024.07.21.24310775
Mohammad Alkhalaf, Chao Deng, Jun Shen, Hui-Chen (Rita) Chang, Ping Yu
Purpose: Malnutrition is a serious health concern, particularly among the older people living in residential aged care facilities. An automated and efficient method is required to identify the individuals afflicted with malnutrition in this setting. The recent advancements in transformer-based large language models (LLMs) equipped with sophisticated context-aware embeddings, such as RoBERTa, have significantly improved machine learning performance, particularly in predictive modelling. Enhancing the embeddings of these models on domain-specific corpora, such as clinical notes, is essential for elevating their performance in clinical tasks. Therefore, our study introduces a novel approach that trains a foundational RoBERTa model on nursing progress notes to develop a RAC domain-specific LLM. The model is further fine-tuned on nursing progress notes to enhance malnutrition identification and prediction in residential aged care setting. Methods: We develop our domain-specific model by training the RoBERTa LLM on 500,000 nursing progress notes from residential aged care electronic health records (EHRs). The model embeddings were used for two downstream tasks: malnutrition note identification and malnutrition prediction. Its performance was compared against baseline RoBERTa and BioClinicalBERT. Furthermore, we truncated long sequence text to fit into RoBERTa 512-token sequence length limitation, enabling our model to handle sequences up to1536 tokens. Results: Utilizing 5-fold cross-validation for both tasks, our RAC domain-specific LLM demonstrated significantly better performance over other models. In malnutrition note identification, it achieved a slightly higher F1-score of 0.966 compared to other LLMs. In prediction, it achieved significantly higher F1-score of 0.655. We enhanced our model predictive capability by integrating the risk factors extracted from each client notes, creating a combined data layer of structured risk factors and free-text notes. This integration improved the prediction performance, evidenced by an increased F1-score of 0.687. Conclusion: Our findings suggest that further fine-tuning a large language model on a domain-specific clinical corpus can improve the foundational model performance in clinical tasks. This specialized adaptation significantly improves our domain-specific model performance in tasks such as malnutrition risk identification and malnutrition prediction, making it useful for identifying and predicting malnutrition among older people living in residential aged care or long-term care facilities.
目的：营养不良是一个严重的健康问题，尤其是居住在养老院的老年人。在这种情况下，需要一种自动高效的方法来识别营养不良者。基于转换器的大型语言模型（LLM）配备了复杂的上下文感知嵌入（如 RoBERTa），其最新进展显著提高了机器学习性能，尤其是在预测建模方面。加强这些模型在特定领域语料库（如临床笔记）中的嵌入对于提高它们在临床任务中的性能至关重要。因此，我们的研究引入了一种新方法，即在护理进展笔记上训练基础 RoBERTa 模型，从而开发出针对特定领域的 RAC LLM。该模型在护理进展记录的基础上进行了进一步的微调，以提高住院养老护理环境中营养不良的识别和预测能力：方法：我们通过对来自养老院电子健康记录（EHR）的 500,000 份护理进展记录训练 RoBERTa LLM，从而开发出针对特定领域的模型。模型嵌入用于两个下游任务：营养不良记录识别和营养不良预测。我们将其性能与基线 RoBERTa 和 BioClinicalBERT 进行了比较。此外，我们对长序列文本进行了截断，以适应 RoBERTa 512 个标记的序列长度限制，从而使我们的模型能够处理多达 1536 个标记的序列：通过对这两项任务进行 5 倍交叉验证，我们的 RAC 特定领域 LLM 的性能明显优于其他模型。在营养不良注释识别方面，与其他 LLM 相比，它的 F1 分数略高，为 0.966。在预测方面，它的 F1 分数明显更高，达到 0.655。我们通过整合从每个客户笔记中提取的风险因素，创建了结构化风险因素和自由文本笔记的组合数据层，从而增强了模型的预测能力。这种整合提高了预测性能，F1 分数提高到了 0.687：我们的研究结果表明，在特定领域的临床语料库上进一步微调大型语言模型可以提高基础模型在临床任务中的性能。在营养不良风险识别和营养不良预测等任务中，这种专业化的调整大大提高了特定领域模型的性能，使其可用于识别和预测居住在养老院或长期护理机构的老年人的营养不良情况。
{"title":"Fine-tuning large language models for effective nutrition support in residential aged care: a domain expertise approach","authors":"Mohammad Alkhalaf, Chao Deng, Jun Shen, Hui-Chen (Rita) Chang, Ping Yu","doi":"10.1101/2024.07.21.24310775","DOIUrl":"https://doi.org/10.1101/2024.07.21.24310775","url":null,"abstract":"Purpose: Malnutrition is a serious health concern, particularly among the older people living in residential aged care facilities. An automated and efficient method is required to identify the individuals afflicted with malnutrition in this setting. The recent advancements in transformer-based large language models (LLMs) equipped with sophisticated context-aware embeddings, such as RoBERTa, have significantly improved machine learning performance, particularly in predictive modelling. Enhancing the embeddings of these models on domain-specific corpora, such as clinical notes, is essential for elevating their performance in clinical tasks. Therefore, our study introduces a novel approach that trains a foundational RoBERTa model on nursing progress notes to develop a RAC domain-specific LLM. The model is further fine-tuned on nursing progress notes to enhance malnutrition identification and prediction in residential aged care setting.\u0000Methods: We develop our domain-specific model by training the RoBERTa LLM on 500,000 nursing progress notes from residential aged care electronic health records (EHRs). The model embeddings were used for two downstream tasks: malnutrition note identification and malnutrition prediction. Its performance was compared against baseline RoBERTa and BioClinicalBERT. Furthermore, we truncated long sequence text to fit into RoBERTa 512-token sequence length limitation, enabling our model to handle sequences up to1536 tokens.\u0000Results: Utilizing 5-fold cross-validation for both tasks, our RAC domain-specific LLM demonstrated significantly better performance over other models. In malnutrition note identification, it achieved a slightly higher F1-score of 0.966 compared to other LLMs. In prediction, it achieved significantly higher F1-score of 0.655. We enhanced our model predictive capability by integrating the risk factors extracted from each client notes, creating a combined data layer of structured risk factors and free-text notes. This integration improved the prediction performance, evidenced by an increased F1-score of 0.687.\u0000Conclusion: Our findings suggest that further fine-tuning a large language model on a domain-specific clinical corpus can improve the foundational model performance in clinical tasks. This specialized adaptation significantly improves our domain-specific model performance in tasks such as malnutrition risk identification and malnutrition prediction, making it useful for identifying and predicting malnutrition among older people living in residential aged care or long-term care facilities.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"15 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141744327","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-19DOI: 10.1101/2024.07.19.24310695
Katerina D. Argyri, Ioannis K. Gallos, Angelos Amditis, Dimitra D. Dionysiou
Cardiovascular disease has been established as the world's number one killer, causing over 20 million deaths per year. This fact, along with the growing awareness of the impact of exposomic risk factors on cardiovascular diseases, has led the scientific community to leverage machine learning strategies as a complementary approach to traditional statistical epidemiological studies that are challenged by the highly heterogeneous and dynamic nature of exposomics data. The principal objective served by this work is to identify key pertinent literature and provide an overview of the breadth of research in the field of machine learning applications on exposomics data with a focus on cardiovascular diseases. Secondarily, we aimed at identifying common limitations and meaningful directives to be addressed in the future. Overall, this work shows that, despite the fact that machine learning on exposomics data is under-researched compared to its application on other members of the -omics family, it is increasingly adopted to investigate different aspects of cardiovascular diseases.
{"title":"Exposomics and Cardiovascular Diseases: A Scoping Review of Machine Learning Approaches","authors":"Katerina D. Argyri, Ioannis K. Gallos, Angelos Amditis, Dimitra D. Dionysiou","doi":"10.1101/2024.07.19.24310695","DOIUrl":"https://doi.org/10.1101/2024.07.19.24310695","url":null,"abstract":"Cardiovascular disease has been established as the world's number one killer, causing over 20 million deaths per year. This fact, along with the growing awareness of the impact of exposomic risk factors on cardiovascular diseases, has led the scientific community to leverage machine learning strategies as a complementary approach to traditional statistical epidemiological studies that are challenged by the highly heterogeneous and dynamic nature of exposomics data. The principal objective served by this work is to identify key pertinent literature and provide an overview of the breadth of research in the field of machine learning applications on exposomics data with a focus on cardiovascular diseases. Secondarily, we aimed at identifying common limitations and meaningful directives to be addressed in the future. Overall, this work shows that, despite the fact that machine learning on exposomics data is under-researched compared to its application on other members of the -omics family, it is increasingly adopted to investigate different aspects of cardiovascular diseases.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"35 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141746269","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-19DOI: 10.1101/2024.07.18.24310636
Ralf Müller-Polyzou, Melanie Reuter-Oppermann
Background: The contemporary world is challenged by natural disasters accelerated by climate change, affecting a growing world population. Simultaneously, cancer remains a persistent threat as a leading cause of death, killing 10~million people annually. The efficacy of radiotherapy, a cornerstone in cancer treatment worldwide, depends on an uninterrupted course of therapy. However, natural disasters cause significant disruptions to the continuity of radiotherapy services, posing a critical challenge to cancer treatment. This paper explores how natural disasters impact radiotherapy practice, compares them to man-made disasters, and outlines strategies to mitigate adverse effects of natural disasters. Through this analysis, the study seeks to contribute to developing resilient healthcare frameworks capable of sustaining essential cancer treatment amidst the challenges posed by natural disasters. Method: We conducted a Structured Literature Review to investigate this matter comprehensively, gathering and evaluating relevant academic publications. We explored how natural disasters affected radiotherapy practice and examined the experience of radiotherapy centres worldwide in resuming operations after such events. Subsequently, we validated and extended our research findings through a global online survey involving radiotherapy professionals. Results: The Structured Literature Review identified twelve academic publications describing hurricanes, floods, and earthquakes as the primary disruptors of radiotherapy practice. The analysis confirms and complements risk mitigation themes identified in our previous research, which focused on the continuity of radiotherapy practice during the COVID-19 pandemic. Our work describes nine overarching themes, forming the basis for a taxonomy of 36 distinct groups. The subsequent confirmative online survey supported and solidified our findings and served as a basis for developing a conceptual framework for natural disaster-resilient radiotherapy. Discussion: The growing threat posed by natural disasters underscores the need to develop business continuity programs and define risk mitigation measures to ensure the uninterrupted provision of radiotherapy services. By drawing lessons from past disasters, we can better prepare for future hazards, supporting disaster management and planning efforts, particularly enhancing the resilience of radiotherapy practice. Additionally, our study can serve as a resource for shaping policy initiatives aimed at mitigating the impact of natural hazards.
{"title":"Radiotherapy continuity for cancer treatment: lessons learned from natural disasters","authors":"Ralf Müller-Polyzou, Melanie Reuter-Oppermann","doi":"10.1101/2024.07.18.24310636","DOIUrl":"https://doi.org/10.1101/2024.07.18.24310636","url":null,"abstract":"Background:\u0000The contemporary world is challenged by natural disasters accelerated by climate change, affecting a growing world population. Simultaneously, cancer remains a persistent threat as a leading cause of death, killing 10~million people annually. The efficacy of radiotherapy, a cornerstone in cancer treatment worldwide, depends on an uninterrupted course of therapy. However, natural disasters cause significant disruptions to the continuity of radiotherapy services, posing a critical challenge to cancer treatment. This paper explores how natural disasters impact radiotherapy practice, compares them to man-made disasters, and outlines strategies to mitigate adverse effects of natural disasters. Through this analysis, the study seeks to contribute to developing resilient healthcare frameworks capable of sustaining essential cancer treatment amidst the challenges posed by natural disasters.\u0000Method:\u0000We conducted a Structured Literature Review to investigate this matter comprehensively, gathering and evaluating relevant academic publications. We explored how natural disasters affected radiotherapy practice and examined the experience of radiotherapy centres worldwide in resuming operations after such events. Subsequently, we validated and extended our research findings through a global online survey involving radiotherapy professionals.\u0000Results:\u0000The Structured Literature Review identified twelve academic publications describing hurricanes, floods, and earthquakes as the primary disruptors of radiotherapy practice. The analysis confirms and complements risk mitigation themes identified in our previous research, which focused on the continuity of radiotherapy practice during the COVID-19 pandemic. Our work describes nine overarching themes, forming the basis for a taxonomy of 36 distinct groups. The subsequent confirmative online survey supported and solidified our findings and served as a basis for developing a conceptual framework for natural disaster-resilient radiotherapy.\u0000Discussion:\u0000The growing threat posed by natural disasters underscores the need to develop business continuity programs and define risk mitigation measures to ensure the uninterrupted provision of radiotherapy services. By drawing lessons from past disasters, we can better prepare for future hazards, supporting disaster management and planning efforts, particularly enhancing the resilience of radiotherapy practice. Additionally, our study can serve as a resource for shaping policy initiatives aimed at mitigating the impact of natural hazards.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"43 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141744185","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-19DOI: 10.1101/2024.07.18.24310656
Jason MIsurac, Lindsey A Knake, James M Blum
Background: Healthcare provider burnout is a critical issue with significant implications for individual well-being, patient care, and healthcare system efficiency. Addressing burnout is essential for improving both provider well-being and the quality of patient care. Ambient artificial intelligence (AI) offers a novel approach to mitigating burnout by reducing the documentation burden through advanced speech recognition and natural language processing technologies that summarize the patient encounter into a clinical note to be reviewed by clinicians. Objective: To assess provider burnout and professional fulfilment associated with Ambient AI technology during a pilot study, assessed using the Stanford Professional Fulfillment Index (PFI). Methods: A pre-post observational study was conducted at University of Iowa Health Care with 38 volunteer physicians and advanced practice providers. Participants used a commercial ambient AI tool, over a 5-week trial in ambulatory environments. The AI tool transcribed patient-clinician conversations and generated preliminary clinical notes for review and entry into the electronic medical record. Burnout and professional fulfillment were assessed using the Stanford PFI at baseline and post-intervention. Results: Pre-test and post-test surveys were completed by 35/38 participants (92% survey completion rate). Results showed a significant reduction in burnout scores, with the median burnout score improving from 4.16 to 3.16 (p=0.005), with validated Stanford PFI cutoff for overall burnout 3.33. Burnout rates decreased from 69% to 43%. There was a notable improvement in interpersonal disengagement scores (3.6 vs. 2.5, p<0.001), although work exhaustion scores did not significantly change. Professional fulfillment showed a modest, non-significant increase (6.1 vs. 6.5, p=0.10). Conclusions: Ambient AI significantly reduces healthcare provider burnout and modestly enhances professional fulfillment. By alleviating documentation burdens, ambient AI improves operational efficiency and provider well-being. These findings suggest that broader implementation of ambient AI could be a strategic intervention to combat burnout in healthcare settings.
{"title":"Impact of Ambient Artificial Intelligence Notes on Provider Burnout","authors":"Jason MIsurac, Lindsey A Knake, James M Blum","doi":"10.1101/2024.07.18.24310656","DOIUrl":"https://doi.org/10.1101/2024.07.18.24310656","url":null,"abstract":"Background: Healthcare provider burnout is a critical issue with significant implications for individual well-being, patient care, and healthcare system efficiency. Addressing burnout is essential for improving both provider well-being and the quality of patient care. Ambient artificial intelligence (AI) offers a novel approach to mitigating burnout by reducing the documentation burden through advanced speech recognition and natural language processing technologies that summarize the patient encounter into a clinical note to be reviewed by clinicians.\u0000Objective: To assess provider burnout and professional fulfilment associated with Ambient AI technology during a pilot study, assessed using the Stanford Professional Fulfillment Index (PFI). Methods: A pre-post observational study was conducted at University of Iowa Health Care with 38 volunteer physicians and advanced practice providers. Participants used a commercial ambient AI tool, over a 5-week trial in ambulatory environments. The AI tool transcribed patient-clinician conversations and generated preliminary clinical notes for review and entry into the electronic medical record. Burnout and professional fulfillment were assessed using the Stanford PFI at baseline and post-intervention. Results: Pre-test and post-test surveys were completed by 35/38 participants (92% survey completion rate). Results showed a significant reduction in burnout scores, with the median burnout score improving from 4.16 to 3.16 (p=0.005), with validated Stanford PFI cutoff for overall burnout 3.33. Burnout rates decreased from 69% to 43%. There was a notable improvement in interpersonal disengagement scores (3.6 vs. 2.5, p<0.001), although work exhaustion scores did not significantly change. Professional fulfillment showed a modest, non-significant increase (6.1 vs. 6.5, p=0.10). Conclusions: Ambient AI significantly reduces healthcare provider burnout and modestly enhances professional fulfillment. By alleviating documentation burdens, ambient AI improves operational efficiency and provider well-being. These findings suggest that broader implementation of ambient AI could be a strategic intervention to combat burnout in healthcare settings.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"116 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141744329","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-19DOI: 10.1101/2024.07.18.24310641
Maximin Lange, Eoin Gogarty, Meredith Martyn, Philip Braude, Feras Fayez, Ben Carter
We will develop a novel approach to drug repurposing, utilising Natural Language Processing (NLP) and Literature Based Discovery (LBD) techniques. This will present a simplified, accessible drug repurposing pipeline using Word2Vec embeddings trained on PubMed abstracts to identify potential new medications to be repurposed. We present this approach in the context of antipsychotics, but it could be repeated for any available medication. The research is structured in three stages: 1. Identification of candidate medications using Word2Vec algorithm trained on scientific literature. 2. Empirical testing of identified candidates using a large hospital dataset to explore protective effects against disease onset. 3. Validation of findings using a second, independent dataset to assess generalizability. This method addresses limitations in current machine learning-based drug repurposing approaches, including lack of external validation and limited accessibility. By leveraging Word2Vec's ability to capture semantic relationships between words, the study aims to uncover hidden connections in medical literature that may lead to novel therapeutic discoveries. The protocol emphasizes transparency and reproducibility, utilizing publicly available electronic health record (EHR) databases for validation. This approach allows for tangible results even for researchers with limited machine learning expertise, bridging the gap between biomedical and information systems communities.
{"title":"Protocol for: A Simple, Accessible, Literature-based Drug Repurposing Pipeline","authors":"Maximin Lange, Eoin Gogarty, Meredith Martyn, Philip Braude, Feras Fayez, Ben Carter","doi":"10.1101/2024.07.18.24310641","DOIUrl":"https://doi.org/10.1101/2024.07.18.24310641","url":null,"abstract":"We will develop a novel approach to drug repurposing, utilising Natural Language Processing (NLP) and Literature Based Discovery (LBD) techniques. This will present a simplified, accessible drug repurposing pipeline using Word2Vec embeddings trained on PubMed abstracts to identify potential new medications to be repurposed. We present this approach in the context of antipsychotics, but it could be repeated for any available medication. The research is structured in three stages:\u00001. Identification of candidate medications using Word2Vec algorithm trained on scientific literature.\u00002. Empirical testing of identified candidates using a large hospital dataset to explore protective effects against disease onset.\u00003. Validation of findings using a second, independent dataset to assess generalizability. This method addresses limitations in current machine learning-based drug repurposing approaches, including lack of external validation and limited accessibility. By leveraging Word2Vec's ability to capture semantic relationships between words, the study aims to uncover hidden connections in medical literature that may lead to novel therapeutic discoveries. The protocol emphasizes transparency and reproducibility, utilizing publicly available electronic health record (EHR) databases for validation. This approach allows for tangible results even for researchers with limited machine learning expertise, bridging the gap between biomedical and information systems communities.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"35 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141744330","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-19DOI: 10.1101/2024.07.18.24310578
Eden Caroline Daniel, SANTOSH TIRUNAGARI, Karan Batth, David Windridge, Yashaswini Balla
Background: Machine learning (ML) prediction of clinically isolated syndrome (CIS) conversion to multiple sclerosis (MS) could be used as a remote, preliminary tool by clinicians to identify high-risk patients that would benefit from early treatment. Objective: This study evaluates ML models to predict CIS to MS conversion and identifies key predictors. Methods: Five supervised learning techniques (Naive Bayes, Logistic Regression, Decision Trees, Random Forests and Support Vector Machines) were applied to clinical data from 138 Lithuanian and 273 Mexican CIS patients. Seven different feature combinations were evaluated to determine the most effective models and predictors. Results: Key predictors common to both datasets included sex, presence of oligoclonal bands in CSF, MRI spinal lesions, abnormal visual evoked potentials and brainstem auditory evoked potentials. The Lithuanian dataset confirmed predictors identified by previous clinical research, while the Mexican dataset partially validated them. The highest F1 score of 1.0 was achieved using Random Forests on all features for the Mexican dataset and Logistic Regression with SMOTE Upsampling on all features for the Lithuanian dataset. Conclusion: Applying the identified high-performing ML models to the CIS patient datasets shows potential in assisting clinicians to identify high-risk patients.
背景：临床孤立综合征（CIS）转化为多发性硬化症（MS）的机器学习（ML）预测可作为一种远程初步工具，供临床医生用于识别可从早期治疗中获益的高风险患者。研究目的本研究评估了预测 CIS 向 MS 转化的 ML 模型，并确定了关键预测因子。方法：将五种监督学习技术（Naive Bayes、逻辑回归、决策树、随机森林和支持向量机）应用于 138 名立陶宛和 273 名墨西哥 CIS 患者的临床数据。对七种不同的特征组合进行了评估，以确定最有效的模型和预测因子。结果：两个数据集共同的关键预测因素包括性别、CSF 中是否存在寡克隆带、MRI 脊柱病变、异常视觉诱发电位和脑干听觉诱发电位。立陶宛数据集证实了之前临床研究确定的预测因子，而墨西哥数据集则部分验证了这些预测因子。在墨西哥数据集的所有特征上使用随机森林，在立陶宛数据集的所有特征上使用逻辑回归和 SMOTE 提升采样，均获得了 1.0 的最高 F1 分数。结论将已确定的高性能 ML 模型应用于 CIS 患者数据集显示出了帮助临床医生识别高风险患者的潜力。
{"title":"Interpretable Machine Learning for Predicting Multiple Sclerosis Conversion from Clinically Isolated Syndrome","authors":"Eden Caroline Daniel, SANTOSH TIRUNAGARI, Karan Batth, David Windridge, Yashaswini Balla","doi":"10.1101/2024.07.18.24310578","DOIUrl":"https://doi.org/10.1101/2024.07.18.24310578","url":null,"abstract":"Background: Machine learning (ML) prediction of clinically isolated syndrome (CIS) conversion to multiple sclerosis (MS) could be used as a remote, preliminary tool by clinicians to identify high-risk patients that would benefit from early treatment. Objective: This study evaluates ML models to predict CIS to MS conversion and identifies key predictors. Methods: Five supervised learning techniques (Naive Bayes, Logistic Regression, Decision Trees, Random Forests and Support Vector Machines) were applied to clinical data from 138 Lithuanian and 273 Mexican CIS patients. Seven different feature combinations were evaluated to determine the most effective models and predictors. Results: Key predictors common to both datasets included sex, presence of oligoclonal bands in CSF, MRI spinal lesions, abnormal visual evoked potentials and brainstem auditory evoked potentials. The Lithuanian dataset confirmed predictors identified by previous clinical research, while the Mexican dataset partially validated them. The highest F1 score of 1.0 was achieved using Random Forests on all features for the Mexican dataset and Logistic Regression with SMOTE Upsampling on all features for the Lithuanian dataset. Conclusion: Applying the identified high-performing ML models to the CIS patient datasets shows potential in assisting clinicians to identify high-risk patients.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"21 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-19","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141746271","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Generative AI (GenAI) has advanced computational pathology through various image translation models. These models synthesize histopathological images from existing ones, facilitating tasks such as color normalization and virtual staining. Current models, while effective, are mostly dedicated to specific source-target domain pairs and lack scalability for multi-domain translations. Here we introduce His-MMDM, a diffusion model-based framework enabling multi-domain and multi-omics histopathological image translation. His-MMDM can translate images across an unlimited number of categorical domains, enabling new applications like the translation of tumor images across various tumor types, while performing comparably to dedicated models on previous tasks such as transforming cryosectioned images to formalin-fixed paraffin-embedded (FFPE) ones. Additionally, it can perform genomics- and/or transcriptomics-guided editing of histopathological images, illustrating the impact of driver mutations and oncogenic pathway alterations on tissue histopathology. These versatile capabilities position His-MMDM as a versatile tool in the GenAI toolkit for future pathologists.
{"title":"His-MMDM: Multi-domain and Multi-omics Translation of Histopathology Images with Diffusion Models","authors":"Zhongxiao Li, Tianqi Su, Bin Zhang, Wenkai Han, Sibin Zhang, Guiyin Sun, Yuwei Cong, Xin Chen, Jiping Qi, Yujie Wang, Shiguang Zhao, Hongxue Meng, Peng Liang, Xin Gao","doi":"10.1101/2024.07.11.24310294","DOIUrl":"https://doi.org/10.1101/2024.07.11.24310294","url":null,"abstract":"Generative AI (GenAI) has advanced computational pathology through various image translation models. These models synthesize histopathological images from existing ones, facilitating tasks such as color normalization and virtual staining. Current models, while effective, are mostly dedicated to specific source-target domain pairs and lack scalability for multi-domain translations. Here we introduce His-MMDM, a diffusion model-based framework enabling multi-domain and multi-omics histopathological image translation. His-MMDM can translate images across an unlimited number of categorical domains, enabling new applications like the translation of tumor images across various tumor types, while performing comparably to dedicated models on previous tasks such as transforming cryosectioned images to formalin-fixed paraffin-embedded (FFPE) ones. Additionally, it can perform genomics- and/or transcriptomics-guided editing of histopathological images, illustrating the impact of driver mutations and oncogenic pathway alterations on tissue histopathology. These versatile capabilities position His-MMDM as a versatile tool in the GenAI toolkit for future pathologists.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141609847","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-12DOI: 10.1101/2024.07.12.24310136
Jakub Jan Dylag, Zlatko Zlatev, Michael Boniface
In clinical research, there is a strong drive to leverage big data from population cohort studies and routine electronic healthcare records to design new interventions, improve health outcomes and increase efficiency of healthcare delivery. Yet, realising these potential demands requires substantial efforts in harmonising source datasets and curating study data, which currently relies on costly and time-consuming manual and labour-intensive methods. We evaluate the applicability of AI methods for natural language processing (NLP) and unsupervised machine learning (ML) to the challenges of big data semantic harmonisation and curation. Our aim is to establish an efficient and robust technological foundation for the development of automated tools supporting data curation of large clinical datasets. We assess NLP and unsupervised ML algorithms and propose two pipelines for automated semantic harmonisation: a pipeline for semantics-aware search for domain relevant variables and a pipeline for clustering of semantically similar variables. We evaluate pipeline performance using 94,037 textual variable descriptions from the English Longitudinal Study of Ageing (ELSA) database. We observe high accuracy of our Semantic Search pipeline with an AUC of 0.899 (SD=0.056). Our Semantic Clustering pipeline achieves a V-measure of 0.237 (SD=0.157), which is on par with leading implementations in other relevant domains. Automation can significantly accelerate the process of dataset harmonization. Manual labelling was performed at a speed of 2.1 descriptions per minute, with our automated labelling increasing speed to 245 descriptions per minute. Our study findings underscore the potential of AI technologies, such as NLP and unsupervised ML, in automating the harmonisation and curation of big data for clinical research. By establishing a robust technological foundation, we pave the way for the development of automated tools that streamline the process, enabling health data scientists to leverage big data more efficiently and effectively in their studies, accelerating insights from data for clinical benefit.
在临床研究中，人们强烈希望利用来自人群队列研究和常规电子医疗记录的大数据来设计新的干预措施、改善健康结果并提高医疗服务效率。然而，要实现这些潜在需求，需要在协调源数据集和整理研究数据方面付出巨大努力，而这目前依赖于成本高、耗时长的人工和劳动密集型方法。我们评估了自然语言处理（NLP）和无监督机器学习（ML）的人工智能方法在应对大数据语义协调和整理挑战方面的适用性。我们的目标是为开发支持大型临床数据集数据整理的自动化工具奠定高效稳健的技术基础。我们对 NLP 算法和无监督 ML 算法进行了评估，并提出了两个用于自动语义协调的管道：一个用于对领域相关变量进行语义感知搜索的管道和一个用于对语义相似变量进行聚类的管道。我们使用英语老龄化纵向研究（ELSA）数据库中的 94,037 个文本变量描述来评估管道性能。我们发现语义搜索管道的准确度很高，AUC为0.899（SD=0.056）。我们的语义聚类管道实现了0.237（SD=0.157）的V-measure，与其他相关领域的领先实现相当。自动化可以大大加快数据集协调过程。人工标注的速度为每分钟 2.1 条描述，而我们的自动标注速度提高到了每分钟 245 条描述。我们的研究结果凸显了 NLP 和无监督 ML 等人工智能技术在临床研究大数据自动协调和整理方面的潜力。通过建立强大的技术基础，我们为开发简化流程的自动化工具铺平了道路，使健康数据科学家能够在研究中更高效、更有效地利用大数据，加快从数据中获得临床益处的洞察力。
{"title":"Pretrained Language Models for Semantics-Aware Data Harmonisation of Observational Clinical Studies in the Era of Big Data","authors":"Jakub Jan Dylag, Zlatko Zlatev, Michael Boniface","doi":"10.1101/2024.07.12.24310136","DOIUrl":"https://doi.org/10.1101/2024.07.12.24310136","url":null,"abstract":"In clinical research, there is a strong drive to leverage big data from population cohort studies and routine electronic healthcare records to design new interventions, improve health outcomes and increase efficiency of healthcare delivery. Yet, realising these potential demands requires substantial efforts in harmonising source datasets and curating study data, which currently relies on costly and time-consuming manual and labour-intensive methods. We evaluate the applicability of AI methods for natural language processing (NLP) and unsupervised machine learning (ML) to the challenges of big data semantic harmonisation and curation. Our aim is to establish an efficient and robust technological foundation for the development of automated tools supporting data curation of large clinical datasets. We assess NLP and unsupervised ML algorithms and propose two pipelines for automated semantic harmonisation: a pipeline for semantics-aware search for domain relevant variables and a pipeline for clustering of semantically similar variables. We evaluate pipeline performance using 94,037 textual variable descriptions from the English Longitudinal Study of Ageing (ELSA) database. We observe high accuracy of our Semantic Search pipeline with an AUC of 0.899 (SD=0.056). Our Semantic Clustering pipeline achieves a V-measure of 0.237 (SD=0.157), which is on par with leading implementations in other relevant domains. Automation can significantly accelerate the process of dataset harmonization. Manual labelling was performed at a speed of 2.1 descriptions per minute, with our automated labelling increasing speed to 245 descriptions per minute. Our study findings underscore the potential of AI technologies, such as NLP and unsupervised ML, in automating the harmonisation and curation of big data for clinical research. By establishing a robust technological foundation, we pave the way for the development of automated tools that streamline the process, enabling health data scientists to leverage big data more efficiently and effectively in their studies, accelerating insights from data for clinical benefit.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"70 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141609845","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-07-12DOI: 10.1101/2024.07.11.24310289
Hadi Amiri, Nidhi Vakil, Mohamed Elgaar, Jiali Cheng, Mitra Mohtarami, Adrian Wong, Mehrnaz Sadrolashrafi, Leo Anthony G. Celi
Abstract Importance: Detecting potential disparities in documented medical decisions is a crucial step toward achieving more equitable practices and care, informing healthcare policy making, and preventing computational models from learning and perpetuating such biases. Objective: To identify disparities associated with race, sex and language proficiency of patients in the documentation of medical decisions. Design: This cross-sectional study included 451 discharge summaries from MIMIC-III, with all medical decisions annotated by domain experts according to the 10 medical decision categories defined in the Decision Identification and Classification Taxonomy for Use in Medicine. Annotated discharge summaries were stratified by race, sex, language proficiency, diagnosis codes, type of ICU, patient status code, and patient comorbidities (quantified by Elixhauser Comorbidity Index) to account for potential confounding factors. Welch's t-test with Bonferroni correction was used to identify significant disparities in the frequency of medical decisions. Setting: The study used the MIMIC-III data set, which contains de-identified health data for patients admitted to the critical care units at the Beth Israel Deaconess Medical Center. Participants: The population reflects the race, sex, and clinical conditions of patients in a data set developed by previous work for patient phenotyping. Main Outcomes and Measures: The primary outcomes were different types of disparities associated with language proficiency of patients in documented medical decisions within discharge summaries, and the secondary outcome was the prevalence of medical decisions documented in discharge summaries. The data set will be made available at https://physionet.org/ Results: This study analyzed 56,759 medical decision text segments documented in 451 discharge summaries. Analysis across demographic groups revealed a higher documentation frequency for English proficient patients compared to non-English proficient patients in several categories, suggesting potential disparities in documentation or care. Specifically, English proficient patients consistently had more documented decisions in critical decision categories such as "Defining Problem" in conditions related to circulatory system and endocrine, nutritional and metabolic diseases. However, this study found no significant disparities in medical decision documentation based on sex or race. Conclusions and Relevance: This study illustrates disparities in the documentation of medical decisions, with English proficient patients receiving more comprehensive documentation compared to non-English proficient patients. Conversely, no significant disparity was identified in terms of sex or race. These findings suggest a potential need for targeted interventions to improve the equity of medical documentation practices so that all patients receive the same level of detailed care documentation and prevent computational models from learning and
{"title":"Analysis of Race, Sex, and Language Proficiency Disparities in Documented Medical Decisions","authors":"Hadi Amiri, Nidhi Vakil, Mohamed Elgaar, Jiali Cheng, Mitra Mohtarami, Adrian Wong, Mehrnaz Sadrolashrafi, Leo Anthony G. Celi","doi":"10.1101/2024.07.11.24310289","DOIUrl":"https://doi.org/10.1101/2024.07.11.24310289","url":null,"abstract":"Abstract\u0000Importance: Detecting potential disparities in documented medical decisions is a crucial step toward achieving more equitable practices and care, informing healthcare policy making, and preventing computational models from learning and perpetuating such biases. Objective: To identify disparities associated with race, sex and language proficiency of patients in the documentation of medical decisions. Design: This cross-sectional study included 451 discharge summaries from MIMIC-III, with all medical decisions annotated by domain experts according to the 10 medical decision categories defined in the Decision Identification and Classification Taxonomy for Use in Medicine. Annotated discharge summaries were stratified by race, sex, language proficiency, diagnosis codes, type of ICU, patient status code, and patient comorbidities (quantified by Elixhauser Comorbidity Index) to account for potential confounding factors. Welch's t-test with Bonferroni correction was used to identify significant disparities in the frequency of medical decisions. Setting: The study used the MIMIC-III data set, which contains de-identified health data for patients admitted to the critical care units at the Beth Israel Deaconess Medical Center. Participants: The population reflects the race, sex, and clinical conditions of patients in a data set developed by previous work for patient phenotyping. Main Outcomes and Measures: The primary outcomes were different types of disparities associated with language proficiency of patients in documented medical decisions within discharge summaries, and the secondary outcome was the prevalence of medical decisions documented in discharge summaries. The data set will be made available at https://physionet.org/ Results: This study analyzed 56,759 medical decision text segments documented in 451 discharge summaries. Analysis across demographic groups revealed a higher documentation frequency for English proficient patients compared to non-English proficient patients in several categories, suggesting potential disparities in documentation or care. Specifically, English proficient patients consistently had more documented decisions in critical decision categories such as \"Defining Problem\" in conditions related to circulatory system and endocrine, nutritional and metabolic diseases. However, this study found no significant disparities in medical decision documentation based on sex or race. Conclusions and Relevance: This study illustrates disparities in the documentation of medical decisions, with English proficient patients receiving more comprehensive documentation compared to non-English proficient patients. Conversely, no significant disparity was identified in terms of sex or race. These findings suggest a potential need for targeted interventions to improve the equity of medical documentation practices so that all patients receive the same level of detailed care documentation and prevent computational models from learning and ","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"13 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141609786","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Background: Mechanical ventilation (MV) is vital for critically ill ICU patients but carries significant mortality risks. This study aims to develop a predictive model to estimate hospital mortality among MV patients, utilizing comprehensive health data to assist ICU physicians with early-stage alerts. Methods: We developed a Machine Learning (ML) framework to predict hospital mortality in ICU patients receiving MV. Using the MIMIC-III database, we identified 25,202 eligible patients through ICD-9 codes. We employed backward elimination and the Lasso method, selecting 32 features based on clinical insights and literature. Data preprocessing included eliminating columns with over 90% missing data and using mean imputation for the remaining missing values. To address class imbalance, we used the Synthetic Minority Over-sampling Technique (SMOTE). We evaluated several ML models, including CatBoost, XGBoost, Decision Tree, Random Forest, Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Logistic Regression, using a 70/30 train-test split. The CatBoost model was chosen for its superior performance in terms of accuracy, precision, recall, F1-score, AUROC metrics, and calibration plots. Results: The study involved a cohort of 25,202 patients on MV. The CatBoost model attained an AUROC of 0.862, an increase from an initial AUROC of 0.821, which was the best reported in the literature. It also demonstrated an accuracy of 0.789, an F1-score of 0.747, and better calibration, outperforming other models. These improvements are due to systematic feature selection and the robust gradient boosting architecture of CatBoost. Conclusion: The preprocessing methodology significantly reduced the number of relevant features, simplifying computational processes, and identified critical features previously overlooked. Integrating these features and tuning the parameters, our model demonstrated strong generalization to unseen data. This highlights the potential of ML as a crucial tool in ICUs, enhancing resource allocation and providing more personalized interventions for MV patients.
{"title":"A Machine Learning-Based Prediction of Hospital Mortality in Mechanically Ventilated ICU Patients","authors":"Hexin Li, Negin Ashrafi, Chris Kang, Guanlan Zhao, Yubing Chen, Maryam Pishgar","doi":"10.1101/2024.07.12.24310325","DOIUrl":"https://doi.org/10.1101/2024.07.12.24310325","url":null,"abstract":"Background:\u0000Mechanical ventilation (MV) is vital for critically ill ICU patients but carries significant mortality risks. This study aims to develop a predictive model to estimate hospital mortality among MV patients, utilizing comprehensive health data to assist ICU physicians with early-stage alerts. Methods:\u0000We developed a Machine Learning (ML) framework to predict hospital mortality in ICU patients receiving MV. Using the MIMIC-III database, we identified 25,202 eligible patients through ICD-9 codes. We employed backward elimination and the Lasso method, selecting 32 features based on clinical insights and literature. Data preprocessing included eliminating columns with over 90% missing data and using mean imputation for the remaining missing values. To address class imbalance, we used the Synthetic Minority Over-sampling Technique (SMOTE). We evaluated several ML models, including CatBoost, XGBoost, Decision Tree, Random Forest, Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Logistic Regression, using a 70/30 train-test split. The CatBoost model was chosen for its superior performance in terms of accuracy, precision, recall, F1-score, AUROC metrics, and calibration plots. Results:\u0000The study involved a cohort of 25,202 patients on MV. The CatBoost model attained an AUROC of 0.862, an increase from an initial AUROC of 0.821, which was the best reported in the literature. It also demonstrated an accuracy of 0.789, an F1-score of 0.747, and better calibration, outperforming other models. These improvements are due to systematic feature selection and the robust gradient boosting architecture of CatBoost. Conclusion:\u0000The preprocessing methodology significantly reduced the number of relevant features, simplifying computational processes, and identified critical features previously overlooked. Integrating these features and tuning the parameters, our model demonstrated strong generalization to unseen data. This highlights the potential of ML as a crucial tool in ICUs, enhancing resource allocation and providing more personalized interventions for MV patients.","PeriodicalId":501454,"journal":{"name":"medRxiv - Health Informatics","volume":"13 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2024-07-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141609850","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：