Kevin G Buell, Kyle A Carey, Nicole Dussault, William F Parker, Jay Dumanian, Sivasubramanium V Bhavani, Emily R Gilbert, Christopher J Winslow, Nirav S Shah, Majid Afshar, Dana P Edelson, Matthew M Churpek
{"title":"Development and Validation of a Machine Learning Model for Early Detection of Untreated Infection.","authors":"Kevin G Buell, Kyle A Carey, Nicole Dussault, William F Parker, Jay Dumanian, Sivasubramanium V Bhavani, Emily R Gilbert, Christopher J Winslow, Nirav S Shah, Majid Afshar, Dana P Edelson, Matthew M Churpek","doi":"10.1097/CCE.0000000000001165","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Early diagnostic uncertainty for infection causes delays in antibiotic administration in infected patients and unnecessary antibiotic administration in noninfected patients.</p><p><strong>Objective: </strong>To develop a machine learning model for the early detection of untreated infection (eDENTIFI), with the presence of infection determined by clinician chart review.</p><p><strong>Derivation cohort: </strong>Three thousand three hundred fifty-seven adult patients hospitalized between 2006 and 2018 at two health systems in Illinois, United States.</p><p><strong>Validation cohort: </strong>We validated in 1632 patients in a third Illinois health system using area under the receiver operating characteristic curve (AUC).</p><p><strong>Prediction model: </strong>Using a longitudinal discrete-time format, we trained a gradient boosted machine model to predict untreated infection in the next 6 hours using routinely available patient demographics, vital signs, and laboratory results.</p><p><strong>Results: </strong>eDENTIFI had an AUC of 0.80 (95% CI, 0.79-0.81) in the validation cohort and outperformed the systemic inflammatory response syndrome criteria with an AUC of 0.64 (95% CI, 0.64-0.65; p < 0.001). The most important features were body mass index, age, temperature, and heart rate. Using a threshold with a 47.6% sensitivity, eDENTIFI detected infection a median 2.0 hours (interquartile range, 0.9-5.2 hr) before antimicrobial administration, with a negative predictive value of 93.6%. Antibiotic administration guided by eDENTIFI could have decreased unnecessary IV antibiotic administration in noninfected patients by 10.8% absolute or 46.4% relative percentage points compared with clinicians.</p><p><strong>Conclusion: </strong>eDENTIFI could both decrease the time to antimicrobial administration in infected patients and unnecessary antibiotic administration in noninfected patients. Further prospective validation is needed.</p>","PeriodicalId":93957,"journal":{"name":"Critical care explorations","volume":"6 10","pages":"e1165"},"PeriodicalIF":0.0000,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11473064/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Critical care explorations","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1097/CCE.0000000000001165","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/10/1 0:00:00","PubModel":"eCollection","JCR":"Q4","JCRName":"Medicine","Score":null,"Total":0}
引用次数: 0
Abstract
Background: Early diagnostic uncertainty for infection causes delays in antibiotic administration in infected patients and unnecessary antibiotic administration in noninfected patients.
Objective: To develop a machine learning model for the early detection of untreated infection (eDENTIFI), with the presence of infection determined by clinician chart review.
Derivation cohort: Three thousand three hundred fifty-seven adult patients hospitalized between 2006 and 2018 at two health systems in Illinois, United States.
Validation cohort: We validated in 1632 patients in a third Illinois health system using area under the receiver operating characteristic curve (AUC).
Prediction model: Using a longitudinal discrete-time format, we trained a gradient boosted machine model to predict untreated infection in the next 6 hours using routinely available patient demographics, vital signs, and laboratory results.
Results: eDENTIFI had an AUC of 0.80 (95% CI, 0.79-0.81) in the validation cohort and outperformed the systemic inflammatory response syndrome criteria with an AUC of 0.64 (95% CI, 0.64-0.65; p < 0.001). The most important features were body mass index, age, temperature, and heart rate. Using a threshold with a 47.6% sensitivity, eDENTIFI detected infection a median 2.0 hours (interquartile range, 0.9-5.2 hr) before antimicrobial administration, with a negative predictive value of 93.6%. Antibiotic administration guided by eDENTIFI could have decreased unnecessary IV antibiotic administration in noninfected patients by 10.8% absolute or 46.4% relative percentage points compared with clinicians.
Conclusion: eDENTIFI could both decrease the time to antimicrobial administration in infected patients and unnecessary antibiotic administration in noninfected patients. Further prospective validation is needed.