Fabian Wiesmüller, Martin Baumgartner, Florian Hoffmann, Mahdi Sareban, Gunnar Treff, Josef Niebauer, Günter Schreier, Dieter Hayn
Telehealth systems have shown to facilitate lifestyle changes like an increase in physical activity. Therefore, an easily quantifiable measure of physical activity levels for both assessing a patient's status quo and tracking physical activity development is needed. The aim of this work was to map semi-structured activities reported as type-intensity-duration triplets in the DiabMemory telehealth system to Metabolic Equivalents of Task (METs). The activity data of 947 telehealth patients were analyzed to create a mapping table between type-intensity pairs and MET values from a preexisting compendium. Additionally, the distribution of activity types and resulting MET scores was evaluated. Combining the MET scores with the duration resulted in the quantified activity measure (MET-minutes). A significant difference in the MET-minutes per activity type (p < 0.0001) was identified. In the future, our method of mapping semi-structured data to METs will serve as a support the evaluation of the effectiveness of DiabMemory.
{"title":"Quantification of Heterogeneous Semi-Structured Patient-Reported Physical Activities Derived from a Diabetes Telehealth Service.","authors":"Fabian Wiesmüller, Martin Baumgartner, Florian Hoffmann, Mahdi Sareban, Gunnar Treff, Josef Niebauer, Günter Schreier, Dieter Hayn","doi":"10.3233/SHTI251520","DOIUrl":"https://doi.org/10.3233/SHTI251520","url":null,"abstract":"<p><p>Telehealth systems have shown to facilitate lifestyle changes like an increase in physical activity. Therefore, an easily quantifiable measure of physical activity levels for both assessing a patient's status quo and tracking physical activity development is needed. The aim of this work was to map semi-structured activities reported as type-intensity-duration triplets in the DiabMemory telehealth system to Metabolic Equivalents of Task (METs). The activity data of 947 telehealth patients were analyzed to create a mapping table between type-intensity pairs and MET values from a preexisting compendium. Additionally, the distribution of activity types and resulting MET scores was evaluated. Combining the MET scores with the duration resulted in the quantified activity measure (MET-minutes). A significant difference in the MET-minutes per activity type (p < 0.0001) was identified. In the future, our method of mapping semi-structured data to METs will serve as a support the evaluation of the effectiveness of DiabMemory.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"170-174"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145214791","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Ather Akhlaq, Muhammad Arsam Qazi, Owais Anwar Golra
The global healthcare system is faced with challenges, including an aging population and increasingly growing co-morbidities, communicable and chronic diseases. Enhanced patient care and experience can be achieved by shifting the healthcare industry to digitalization. However, the digitalization of hospitals in low-middle-income countries is still premature compared to developed countries due to multifaceted challenges. This study explores the current state of digitalization of hospitals in a low-middle-income country, Pakistan. Semi-structured interviews with healthcare industry stakeholders were conducted to gain an in-depth understanding. The study's findings revealed the current state of digitalization in Pakistan, highlighting barriers such as inadequate resources, improper hospital classification, lack of data sharing media, and absence of financing facilities, as well as facilitators such as the COVID-19 pandemic and healthcare staff training.
{"title":"A Case Study to Explore Barriers and Facilitators to the Digitalization of Hospitals in Pakistan.","authors":"Ather Akhlaq, Muhammad Arsam Qazi, Owais Anwar Golra","doi":"10.3233/SHTI251558","DOIUrl":"https://doi.org/10.3233/SHTI251558","url":null,"abstract":"<p><p>The global healthcare system is faced with challenges, including an aging population and increasingly growing co-morbidities, communicable and chronic diseases. Enhanced patient care and experience can be achieved by shifting the healthcare industry to digitalization. However, the digitalization of hospitals in low-middle-income countries is still premature compared to developed countries due to multifaceted challenges. This study explores the current state of digitalization of hospitals in a low-middle-income country, Pakistan. Semi-structured interviews with healthcare industry stakeholders were conducted to gain an in-depth understanding. The study's findings revealed the current state of digitalization in Pakistan, highlighting barriers such as inadequate resources, improper hospital classification, lack of data sharing media, and absence of financing facilities, as well as facilitators such as the COVID-19 pandemic and healthcare staff training.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"340-344"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145214822","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
David Fernández-Narro, Pablo Ferri, Juan Miguel García-Gómez, Carlos Sáez
Out-of-distribution data , data coming from a different distribution with respect to the training data, entails a critical challenge for the robustness and safety of AI-based clinical decision support systems (CDSSs). This work aims to investigate whether real-time, sample-level quantification of epistemic uncertainty, the model's uncertainty due to limited knowledge of the true data-generating process, can act as a lightweight safety layer for health AI and CDSSs, targeting model updates and spotlighting human review. To this end, we trained and evaluated a continual learning-based neural network classifier on quarterly batches in a real-world Mexican COVID-19 dataset. For each training window, we estimated the distribution of the prediction epistemic uncertainties using Monte Carlo Dropout. We set a data-driven uncertainty threshold to determine potential out-of-distribution samples at 95% of that distribution. Results across all training-test time pairs show that samples below this threshold exhibit consistently higher macro-F1 and render performance virtually invariant to temporal drift, while the flagged samples captured most prediction errors. Since our method requires no model retraining, sample-level epistemic uncertainty screening offers a practical and efficient first line of defense for deploying health-AI systems in dynamic environments.
{"title":"Quantifying Epistemic Uncertainty in Predictions for Safer Health AI Performance Under Dataset Shifts.","authors":"David Fernández-Narro, Pablo Ferri, Juan Miguel García-Gómez, Carlos Sáez","doi":"10.3233/SHTI251493","DOIUrl":"https://doi.org/10.3233/SHTI251493","url":null,"abstract":"<p><p>Out-of-distribution data , data coming from a different distribution with respect to the training data, entails a critical challenge for the robustness and safety of AI-based clinical decision support systems (CDSSs). This work aims to investigate whether real-time, sample-level quantification of epistemic uncertainty, the model's uncertainty due to limited knowledge of the true data-generating process, can act as a lightweight safety layer for health AI and CDSSs, targeting model updates and spotlighting human review. To this end, we trained and evaluated a continual learning-based neural network classifier on quarterly batches in a real-world Mexican COVID-19 dataset. For each training window, we estimated the distribution of the prediction epistemic uncertainties using Monte Carlo Dropout. We set a data-driven uncertainty threshold to determine potential out-of-distribution samples at 95% of that distribution. Results across all training-test time pairs show that samples below this threshold exhibit consistently higher macro-F1 and render performance virtually invariant to temporal drift, while the flagged samples captured most prediction errors. Since our method requires no model retraining, sample-level epistemic uncertainty screening offers a practical and efficient first line of defense for deploying health-AI systems in dynamic environments.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"47-51"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145214928","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
AI chatbots have introduced a new dimension to how we provide healthcare and support healthcare clinicians. However, despite the benefits of chatbots, their adoption raises ethical concerns related to the effects on users. This study discusses the ethical implications of psychological dependency on autonomy and decision-making in the context of healthcare delivery. Ten studies were analysed, revealing a cascading hierarchy involving the interconnected risks of threats to autonomy, disruption of critical thinking due to over-reliance, and psychological dependency, as well as issues of bias and misinformation in chatbot outputs, limitations in trust and reliability, and a mixed impact on clinicians' well-being. These findings underscore the importance of adopting a balanced approach to integrating AI chatbots into clinical practice, with a strong emphasis on preserving clinical autonomy to maintain the overall well-being of healthcare practitioners.
{"title":"The Illusion of Control: AI Chatbot Dependency and the Threat to Clinical Autonomy.","authors":"Roa'a Aljuraid","doi":"10.3233/SHTI251529","DOIUrl":"https://doi.org/10.3233/SHTI251529","url":null,"abstract":"<p><p>AI chatbots have introduced a new dimension to how we provide healthcare and support healthcare clinicians. However, despite the benefits of chatbots, their adoption raises ethical concerns related to the effects on users. This study discusses the ethical implications of psychological dependency on autonomy and decision-making in the context of healthcare delivery. Ten studies were analysed, revealing a cascading hierarchy involving the interconnected risks of threats to autonomy, disruption of critical thinking due to over-reliance, and psychological dependency, as well as issues of bias and misinformation in chatbot outputs, limitations in trust and reliability, and a mixed impact on clinicians' well-being. These findings underscore the importance of adopting a balanced approach to integrating AI chatbots into clinical practice, with a strong emphasis on preserving clinical autonomy to maintain the overall well-being of healthcare practitioners.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"211-215"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145215005","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Michael Rigby, Elisavet Andrikopoulou, Mirela Prgomet, Stephanie Medlock, Zoie Sy Wong, Kathrin Cresswell
Artificial Intelligence (AI) is a rapidly growing technology within health informatics, but it is not subject to the rigor of scientific and safety validation required for all other new health techniques. Moreover, some functions of health AI cannot only introduce biases but can then reinforce and spread them by building on them. Thus, while health AI may bring benefit, it can also pose risks for safety and efficiency, as end users cannot rely on rigorous pre-implementation evidence or in-use validation. This review aims to revisit the principles and techniques already developed in health informatics, to build scientific principles for AI evaluation and the production of evidence. The Precautionary Principle provides further justification for such processes, and continuous quality improvement methods can add assurance. Developers should be expected to provide a robust evidence and evaluation trail, and clinicians and patient groups should expect this to be required by policy makers. This needs to be balanced with a need for developing pragmatic and agile evaluation methods in this fast-evolving area, to deepen knowledge and to guard against the risk of hidden perpetuation of errors.
{"title":"Validation and Evaluation as Essentials to Ensuring Safe AI Health Applications.","authors":"Michael Rigby, Elisavet Andrikopoulou, Mirela Prgomet, Stephanie Medlock, Zoie Sy Wong, Kathrin Cresswell","doi":"10.3233/SHTI251494","DOIUrl":"10.3233/SHTI251494","url":null,"abstract":"<p><p>Artificial Intelligence (AI) is a rapidly growing technology within health informatics, but it is not subject to the rigor of scientific and safety validation required for all other new health techniques. Moreover, some functions of health AI cannot only introduce biases but can then reinforce and spread them by building on them. Thus, while health AI may bring benefit, it can also pose risks for safety and efficiency, as end users cannot rely on rigorous pre-implementation evidence or in-use validation. This review aims to revisit the principles and techniques already developed in health informatics, to build scientific principles for AI evaluation and the production of evidence. The Precautionary Principle provides further justification for such processes, and continuous quality improvement methods can add assurance. Developers should be expected to provide a robust evidence and evaluation trail, and clinicians and patient groups should expect this to be required by policy makers. This needs to be balanced with a need for developing pragmatic and agile evaluation methods in this fast-evolving area, to deepen knowledge and to guard against the risk of hidden perpetuation of errors.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"52-56"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145215108","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Biometric authentication based on physiological signals offers promising potential for enhancing security in mobile patient monitoring. 'Intelligent medical devices', which check the identity of a patient before usage to address safety risks from device-patient mix-ups, do not yet exist. In this project, an AI-based identification system that uses vital signs for biometric authentication will be realized in order to enable the identification on the basis of biometric patterns. By integrating this component into a patient monitoring platform, a seamless and reliable method for verifying patient identity before device use is established, supporting safer and more efficient clinical workflows.
{"title":"V-IDENT: Enhancing Patient Safety Through PPG-Based User Identification.","authors":"Katja Bochtler, Jonas Schropp, Michael Weber","doi":"10.3233/SHTI251521","DOIUrl":"https://doi.org/10.3233/SHTI251521","url":null,"abstract":"<p><p>Biometric authentication based on physiological signals offers promising potential for enhancing security in mobile patient monitoring. 'Intelligent medical devices', which check the identity of a patient before usage to address safety risks from device-patient mix-ups, do not yet exist. In this project, an AI-based identification system that uses vital signs for biometric authentication will be realized in order to enable the identification on the basis of biometric patterns. By integrating this component into a patient monitoring platform, a seamless and reliable method for verifying patient identity before device use is established, supporting safer and more efficient clinical workflows.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"175-179"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145215121","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Hospital at home (HaH) models involve treating patients at home for conditions that typically require hospitalisation. This paper reports on an expert survey to validate patient selection criteria from the literature. Feedback from 20 experts led to consensus on four criteria: medical condition, clinical suitability, living conditions and social support. No consensus was reached on the criteria demographics, technological readiness and literacy. Five other characteristics were identified. These criteria emphasise the importance of selecting patients on the basis of clinical need, safety, and ability to receive care at home, while taking into account potential inequalities. Future efforts should focus on improving digital readiness, integrating multidisciplinary perspectives, and ensuring equitable access to HaH services.
{"title":"Exploring the Relevance of Patient Selection Criteria for Hospital at Home Care: Results from an Expert Survey.","authors":"Kerstin Denecke, Octavio Rivera-Romero","doi":"10.3233/SHTI251502","DOIUrl":"https://doi.org/10.3233/SHTI251502","url":null,"abstract":"<p><p>Hospital at home (HaH) models involve treating patients at home for conditions that typically require hospitalisation. This paper reports on an expert survey to validate patient selection criteria from the literature. Feedback from 20 experts led to consensus on four criteria: medical condition, clinical suitability, living conditions and social support. No consensus was reached on the criteria demographics, technological readiness and literacy. Five other characteristics were identified. These criteria emphasise the importance of selecting patients on the basis of clinical need, safety, and ability to receive care at home, while taking into account potential inequalities. Future efforts should focus on improving digital readiness, integrating multidisciplinary perspectives, and ensuring equitable access to HaH services.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"88-92"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145215128","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Mathieu Beaudeau, Nicolas Nesseler, Jean-Philippe Verhoye, Erwan Flecher, Marc Cuggia, Boris Delange
Extracorporeal Membrane Oxygenation (ECMO) is a life-saving cardiopulmonary support for patients with acute heart failure. However, the process of weaning from veno-arterial (V-A) ECMO remains complex and risky. We developed a machine learning-based predictive model to assist clinicians in identifying patients with a high probability of successful weaning. This retrospective monocentric study included 122 patients admitted to Rennes University Hospital between January 2020 and January 2023. Data from the eHOP clinical data warehouse were used to train and evaluate various machine learning algorithms, including Random Forest, XGBoost, KNN, SVM, and regularized logistic regressions. The best-performing models showed an AUC of 0.84-0.86, with XGBoost offering the highest results (0.86 [0.72-0.96]). Key predictors included ECMO flow rate, oxygenation fraction (FmO2), and duration of ECMO. While these results are promising, further validation is required before such tools can be translated into clinical decision-making processes.
{"title":"Predicting Successful Weaning from Veno-Arterial ECMO Using Machine Learning.","authors":"Mathieu Beaudeau, Nicolas Nesseler, Jean-Philippe Verhoye, Erwan Flecher, Marc Cuggia, Boris Delange","doi":"10.3233/SHTI251489","DOIUrl":"https://doi.org/10.3233/SHTI251489","url":null,"abstract":"<p><p>Extracorporeal Membrane Oxygenation (ECMO) is a life-saving cardiopulmonary support for patients with acute heart failure. However, the process of weaning from veno-arterial (V-A) ECMO remains complex and risky. We developed a machine learning-based predictive model to assist clinicians in identifying patients with a high probability of successful weaning. This retrospective monocentric study included 122 patients admitted to Rennes University Hospital between January 2020 and January 2023. Data from the eHOP clinical data warehouse were used to train and evaluate various machine learning algorithms, including Random Forest, XGBoost, KNN, SVM, and regularized logistic regressions. The best-performing models showed an AUC of 0.84-0.86, with XGBoost offering the highest results (0.86 [0.72-0.96]). Key predictors included ECMO flow rate, oxygenation fraction (FmO2), and duration of ECMO. While these results are promising, further validation is required before such tools can be translated into clinical decision-making processes.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"27-31"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145215148","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The rise in infectious waste during the COVID-19 pandemic exposed critical challenges in Thailand's waste management systems, particularly within sub-district public health facilities. This study aimed to develop and implement an infectious waste management database system for 14 Sub-District Health Promoting Hospitals (HPHs) in Kantharawichai District, Maha Sarakham Province. Using a Research and Development (R&D) model and the knowledge-attitudes-practices (KAP) model to understand behaviors. The development phase engaged 145 community caregivers, of whom 95.17% were female and 74.48% aged between 30-59 years. Results showed that 56.55% of participants had a high knowledge of infectious waste management, while 42.76% expressed a high level of positive attitudes. In terms of behavior, 37.93% demonstrated high compliance with appropriate waste handling practices. Data derived from KAP, and interviews were used as the main inputs to develop the database system. The system included real-time dashboards, GPS-tagged data inputs, automated alerts, and data visualization tools using Microsoft Excel and Power BI. This research offers a scalable digital solution for enhancing infectious waste management, particularly in resource-limited community health settings.
{"title":"Integrating Technology-Driven Database System into Infectious Waste Management for Resource-Limited Settings.","authors":"Niruwan Turnbull, Chamaiphon Phaengtho, Jindawan Wibuloutai, Ruchakron Kongmant, Kannikar Hannah Wechkunanukul","doi":"10.3233/SHTI251515","DOIUrl":"https://doi.org/10.3233/SHTI251515","url":null,"abstract":"<p><p>The rise in infectious waste during the COVID-19 pandemic exposed critical challenges in Thailand's waste management systems, particularly within sub-district public health facilities. This study aimed to develop and implement an infectious waste management database system for 14 Sub-District Health Promoting Hospitals (HPHs) in Kantharawichai District, Maha Sarakham Province. Using a Research and Development (R&D) model and the knowledge-attitudes-practices (KAP) model to understand behaviors. The development phase engaged 145 community caregivers, of whom 95.17% were female and 74.48% aged between 30-59 years. Results showed that 56.55% of participants had a high knowledge of infectious waste management, while 42.76% expressed a high level of positive attitudes. In terms of behavior, 37.93% demonstrated high compliance with appropriate waste handling practices. Data derived from KAP, and interviews were used as the main inputs to develop the database system. The system included real-time dashboards, GPS-tagged data inputs, automated alerts, and data visualization tools using Microsoft Excel and Power BI. This research offers a scalable digital solution for enhancing infectious waste management, particularly in resource-limited community health settings.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"149-153"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145215162","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
STOPP/START v3 is a set of criteria for optimizing therapy for elderly patients with polypharmacy. Implementing these criteria in prescribing software requires to formalize them, which is a difficult task. This project aimed to automate the formalization of these criteria using large language models (LLMs), specifically leveraging Retrieval-Augmented Generation (RAG) for enhanced accuracy. We employed DeepSeek and GPT-4o-mini for entity extraction, code mapping to ICD-10, LOINC, and ATC, and the generation of executable Python code. A preliminary evaluation conducted on a subset of rules yielded a notably high F1-score (0.90, 0.92, 1 for drug, disease and observation entity mapping respectively and perfect results for medical entity extraction and code logic consistency). These results confirm the model's effectiveness in accurately transforming complex clinical rules into executable code. In conclusion, we successfully automated the creation of executable code from medical guidelines, proving that LLMs, supported by RAG, can be effective for automating clinical decision support tasks and formalizing medical rules.
{"title":"From Guidelines to Code: Formalizing STOPP/START Criteria Using LLMs and RAG for Clinical Decision Support.","authors":"Samya Adrouji, Abdelmalek Mouazer, Jean-Baptise Lamy","doi":"10.3233/SHTI251492","DOIUrl":"https://doi.org/10.3233/SHTI251492","url":null,"abstract":"<p><p>STOPP/START v3 is a set of criteria for optimizing therapy for elderly patients with polypharmacy. Implementing these criteria in prescribing software requires to formalize them, which is a difficult task. This project aimed to automate the formalization of these criteria using large language models (LLMs), specifically leveraging Retrieval-Augmented Generation (RAG) for enhanced accuracy. We employed DeepSeek and GPT-4o-mini for entity extraction, code mapping to ICD-10, LOINC, and ATC, and the generation of executable Python code. A preliminary evaluation conducted on a subset of rules yielded a notably high F1-score (0.90, 0.92, 1 for drug, disease and observation entity mapping respectively and perfect results for medical entity extraction and code logic consistency). These results confirm the model's effectiveness in accurately transforming complex clinical rules into executable code. In conclusion, we successfully automated the creation of executable code from medical guidelines, proving that LLMs, supported by RAG, can be effective for automating clinical decision support tasks and formalizing medical rules.</p>","PeriodicalId":94357,"journal":{"name":"Studies in health technology and informatics","volume":"332 ","pages":"42-46"},"PeriodicalIF":0.0,"publicationDate":"2025-10-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"145215167","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}