Alessandro Giaj Levra, Mauro Gatti, Roberto Mene, Dana Shiffer, Giorgio Costantino, Monica Solbiati, Raffaello Furlan, Franca Dipaola
{"title":"基于大语言模型的急诊科晕厥识别临床决策支持系统:临床工作流程整合框架。","authors":"Alessandro Giaj Levra, Mauro Gatti, Roberto Mene, Dana Shiffer, Giorgio Costantino, Monica Solbiati, Raffaello Furlan, Franca Dipaola","doi":"10.1016/j.ejim.2024.09.017","DOIUrl":null,"url":null,"abstract":"<p><p>Differentiation of syncope from transient loss of consciousness can be challenging in the emergency department (ED). Natural Language Processing (NLP) enables the analysis of free text in the electronic medical records (EMR). The present paper aimed to develop a large language models (LLM) for syncope recognition in the ED and proposed a framework for model integration within the clinical workflow. Two models, based on both the Italian and Multilingual Bidirectional Encoder Representations from Transformers (BERT) language model, were developed using consecutive EMRs. The \"triage\" model was only based on notes contained in the \"triage\" section of the EMR. The \"anamnesis\" model added data contained in the \"medical history\" section. Interpretation and calibration plots were generated. The Italian and Multi BERT models were developed and tested on both 15,098 and 15,222 EMRs, respectively. The triage model had an AUC of 0·95 for the Italian BERT and 0·94 for the Multi BERT. The anamnesis model had an AUC of 0·98 for the Italian BERT and 0·97 for Multi BERT. The LLM identified syncope when not explicitly mentioned in the EMR and also recognized common prodromal symptoms preceding syncope. Both models identified syncope patients in the ED with a high discriminative capability from nurses and doctors' notes, thus potentially acting as a tool helping physicians to differentiate syncope from others transient loss of consciousness.</p>","PeriodicalId":50485,"journal":{"name":"European Journal of Internal Medicine","volume":null,"pages":null},"PeriodicalIF":5.9000,"publicationDate":"2024-09-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"A large language model-based clinical decision support system for syncope recognition in the emergency department: A framework for clinical workflow integration.\",\"authors\":\"Alessandro Giaj Levra, Mauro Gatti, Roberto Mene, Dana Shiffer, Giorgio Costantino, Monica Solbiati, Raffaello Furlan, Franca Dipaola\",\"doi\":\"10.1016/j.ejim.2024.09.017\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><p>Differentiation of syncope from transient loss of consciousness can be challenging in the emergency department (ED). Natural Language Processing (NLP) enables the analysis of free text in the electronic medical records (EMR). The present paper aimed to develop a large language models (LLM) for syncope recognition in the ED and proposed a framework for model integration within the clinical workflow. Two models, based on both the Italian and Multilingual Bidirectional Encoder Representations from Transformers (BERT) language model, were developed using consecutive EMRs. The \\\"triage\\\" model was only based on notes contained in the \\\"triage\\\" section of the EMR. The \\\"anamnesis\\\" model added data contained in the \\\"medical history\\\" section. Interpretation and calibration plots were generated. The Italian and Multi BERT models were developed and tested on both 15,098 and 15,222 EMRs, respectively. The triage model had an AUC of 0·95 for the Italian BERT and 0·94 for the Multi BERT. The anamnesis model had an AUC of 0·98 for the Italian BERT and 0·97 for Multi BERT. The LLM identified syncope when not explicitly mentioned in the EMR and also recognized common prodromal symptoms preceding syncope. Both models identified syncope patients in the ED with a high discriminative capability from nurses and doctors' notes, thus potentially acting as a tool helping physicians to differentiate syncope from others transient loss of consciousness.</p>\",\"PeriodicalId\":50485,\"journal\":{\"name\":\"European Journal of Internal Medicine\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":5.9000,\"publicationDate\":\"2024-09-28\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"European Journal of Internal Medicine\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1016/j.ejim.2024.09.017\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"MEDICINE, GENERAL & INTERNAL\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"European Journal of Internal Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1016/j.ejim.2024.09.017","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}
A large language model-based clinical decision support system for syncope recognition in the emergency department: A framework for clinical workflow integration.
Differentiation of syncope from transient loss of consciousness can be challenging in the emergency department (ED). Natural Language Processing (NLP) enables the analysis of free text in the electronic medical records (EMR). The present paper aimed to develop a large language models (LLM) for syncope recognition in the ED and proposed a framework for model integration within the clinical workflow. Two models, based on both the Italian and Multilingual Bidirectional Encoder Representations from Transformers (BERT) language model, were developed using consecutive EMRs. The "triage" model was only based on notes contained in the "triage" section of the EMR. The "anamnesis" model added data contained in the "medical history" section. Interpretation and calibration plots were generated. The Italian and Multi BERT models were developed and tested on both 15,098 and 15,222 EMRs, respectively. The triage model had an AUC of 0·95 for the Italian BERT and 0·94 for the Multi BERT. The anamnesis model had an AUC of 0·98 for the Italian BERT and 0·97 for Multi BERT. The LLM identified syncope when not explicitly mentioned in the EMR and also recognized common prodromal symptoms preceding syncope. Both models identified syncope patients in the ED with a high discriminative capability from nurses and doctors' notes, thus potentially acting as a tool helping physicians to differentiate syncope from others transient loss of consciousness.
期刊介绍:
The European Journal of Internal Medicine serves as the official journal of the European Federation of Internal Medicine and is the primary scientific reference for European academic and non-academic internists. It is dedicated to advancing science and practice in internal medicine across Europe. The journal publishes original articles, editorials, reviews, internal medicine flashcards, and other relevant information in the field. Both translational medicine and clinical studies are emphasized. EJIM aspires to be a leading platform for excellent clinical studies, with a focus on enhancing the quality of healthcare in European hospitals.