Adam Kotter, Samir Abdelrahman, Yi-Ki Jacob Wan, Karl Madaras-Kelly, Keaton L Morgan, Chin Fung Kelvin Kan, Guilherme Del Fiol
{"title":"Improved Interpretability Without Performance Reduction in a Sepsis Prediction Risk Score.","authors":"Adam Kotter, Samir Abdelrahman, Yi-Ki Jacob Wan, Karl Madaras-Kelly, Keaton L Morgan, Chin Fung Kelvin Kan, Guilherme Del Fiol","doi":"10.3390/diagnostics15030307","DOIUrl":null,"url":null,"abstract":"<p><p><b>Objective</b>: Sepsis is a life-threatening response to infection and a major cause of hospital mortality. Machine learning (ML) models have demonstrated better sepsis prediction performance than integer risk scores but are less widely used in clinical settings, in part due to lower interpretability. This study aimed to improve the interpretability of an ML-based model without reducing its performance in non-ICU sepsis prediction. <b>Methods</b>: A logistic regression model was trained to predict sepsis onset and then converted into a more interpretable integer point system, STEWS, using its regression coefficients. We compared STEWS with the logistic regression model using PPV at 90% sensitivity. <b>Results</b>: STEWS was significantly equivalent to logistic regression using the two one-sided tests procedure (0.051 vs. 0.051; <i>p</i> = 0.004). <b>Conclusions</b>: STEWS demonstrated equivalent performance to a comparable logistic regression model for non-ICU sepsis prediction, suggesting that converting ML models into more interpretable forms does not necessarily reduce predictive power.</p>","PeriodicalId":11225,"journal":{"name":"Diagnostics","volume":"15 3","pages":""},"PeriodicalIF":3.3000,"publicationDate":"2025-01-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11817530/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Diagnostics","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.3390/diagnostics15030307","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"MEDICINE, GENERAL & INTERNAL","Score":null,"Total":0}
引用次数: 0
Abstract
Objective: Sepsis is a life-threatening response to infection and a major cause of hospital mortality. Machine learning (ML) models have demonstrated better sepsis prediction performance than integer risk scores but are less widely used in clinical settings, in part due to lower interpretability. This study aimed to improve the interpretability of an ML-based model without reducing its performance in non-ICU sepsis prediction. Methods: A logistic regression model was trained to predict sepsis onset and then converted into a more interpretable integer point system, STEWS, using its regression coefficients. We compared STEWS with the logistic regression model using PPV at 90% sensitivity. Results: STEWS was significantly equivalent to logistic regression using the two one-sided tests procedure (0.051 vs. 0.051; p = 0.004). Conclusions: STEWS demonstrated equivalent performance to a comparable logistic regression model for non-ICU sepsis prediction, suggesting that converting ML models into more interpretable forms does not necessarily reduce predictive power.
目的:脓毒症是一种危及生命的感染反应和医院死亡率的主要原因。机器学习(ML)模型显示出比整数风险评分更好的脓毒症预测性能,但在临床环境中应用较少,部分原因是可解释性较低。本研究旨在提高基于ml的模型的可解释性,同时不降低其在非icu脓毒症预测中的性能。方法:训练逻辑回归模型来预测脓毒症的发作,然后使用其回归系数将其转换为更可解释的整数点系统STEWS。我们将STEWS与使用PPV的逻辑回归模型进行比较,灵敏度为90%。结果:采用两个单侧检验程序进行logistic回归分析,STEWS与logistic回归显著相等(0.051 vs. 0.051;P = 0.004)。结论:在非icu脓毒症预测中,STEWS表现出与可比较的逻辑回归模型相当的性能,这表明将ML模型转换为更可解释的形式并不一定会降低预测能力。
DiagnosticsBiochemistry, Genetics and Molecular Biology-Clinical Biochemistry
CiteScore
4.70
自引率
8.30%
发文量
2699
审稿时长
19.64 days
期刊介绍:
Diagnostics (ISSN 2075-4418) is an international scholarly open access journal on medical diagnostics. It publishes original research articles, reviews, communications and short notes on the research and development of medical diagnostics. There is no restriction on the length of the papers. Our aim is to encourage scientists to publish their experimental and theoretical research in as much detail as possible. Full experimental and/or methodological details must be provided for research articles.