Matthew A Levin, Arash Kia, Prem Timsina, Fu-Yuan Cheng, Kim-Anh-Nhi Nguyen, Roopa Kohli-Seth, Hung-Mo Lin, Yuxia Ouyang, Robert Freeman, David L Reich
{"title":"防止护理升级的实时机器学习警报:非随机分组实用临床试验。","authors":"Matthew A Levin, Arash Kia, Prem Timsina, Fu-Yuan Cheng, Kim-Anh-Nhi Nguyen, Roopa Kohli-Seth, Hung-Mo Lin, Yuxia Ouyang, Robert Freeman, David L Reich","doi":"10.1097/CCM.0000000000006243","DOIUrl":null,"url":null,"abstract":"<p><strong>Objectives: </strong>Machine learning algorithms can outperform older methods in predicting clinical deterioration, but rigorous prospective data on their real-world efficacy are limited. We hypothesized that real-time machine learning generated alerts sent directly to front-line providers would reduce escalations.</p><p><strong>Design: </strong>Single-center prospective pragmatic nonrandomized clustered clinical trial.</p><p><strong>Setting: </strong>Academic tertiary care medical center.</p><p><strong>Patients: </strong>Adult patients admitted to four medical-surgical units. Assignment to intervention or control arms was determined by initial unit admission.</p><p><strong>Interventions: </strong>Real-time alerts stratified according to predicted likelihood of deterioration sent either to the primary team or directly to the rapid response team (RRT). Clinical care and interventions were at the providers' discretion. For the control units, alerts were generated but not sent, and standard RRT activation criteria were used.</p><p><strong>Measurements and main results: </strong>The primary outcome was the rate of escalation per 1000 patient bed days. Secondary outcomes included the frequency of orders for fluids, medications, and diagnostic tests, and combined in-hospital and 30-day mortality. Propensity score modeling with stabilized inverse probability of treatment weight (IPTW) was used to account for differences between groups. Data from 2740 patients enrolled between July 2019 and March 2020 were analyzed (1488 intervention, 1252 control). Average age was 66.3 years and 1428 participants (52%) were female. The rate of escalation was 12.3 vs. 11.3 per 1000 patient bed days (difference, 1.0; 95% CI, -2.8 to 4.7) and IPTW adjusted incidence rate ratio 1.43 (95% CI, 1.16-1.78; p < 0.001). Patients in the intervention group were more likely to receive cardiovascular medication orders (16.1% vs. 11.3%; 4.7%; 95% CI, 2.1-7.4%) and IPTW adjusted relative risk (RR) (1.74; 95% CI, 1.39-2.18; p < 0.001). Combined in-hospital and 30-day-mortality was lower in the intervention group (7% vs. 9.3%; -2.4%; 95% CI, -4.5% to -0.2%) and IPTW adjusted RR (0.76; 95% CI, 0.58-0.99; p = 0.045).</p><p><strong>Conclusions: </strong>Real-time machine learning alerts do not reduce the rate of escalation but may reduce mortality.</p>","PeriodicalId":10765,"journal":{"name":"Critical Care Medicine","volume":null,"pages":null},"PeriodicalIF":7.7000,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Real-Time Machine Learning Alerts to Prevent Escalation of Care: A Nonrandomized Clustered Pragmatic Clinical Trial.\",\"authors\":\"Matthew A Levin, Arash Kia, Prem Timsina, Fu-Yuan Cheng, Kim-Anh-Nhi Nguyen, Roopa Kohli-Seth, Hung-Mo Lin, Yuxia Ouyang, Robert Freeman, David L Reich\",\"doi\":\"10.1097/CCM.0000000000006243\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Objectives: </strong>Machine learning algorithms can outperform older methods in predicting clinical deterioration, but rigorous prospective data on their real-world efficacy are limited. We hypothesized that real-time machine learning generated alerts sent directly to front-line providers would reduce escalations.</p><p><strong>Design: </strong>Single-center prospective pragmatic nonrandomized clustered clinical trial.</p><p><strong>Setting: </strong>Academic tertiary care medical center.</p><p><strong>Patients: </strong>Adult patients admitted to four medical-surgical units. Assignment to intervention or control arms was determined by initial unit admission.</p><p><strong>Interventions: </strong>Real-time alerts stratified according to predicted likelihood of deterioration sent either to the primary team or directly to the rapid response team (RRT). Clinical care and interventions were at the providers' discretion. For the control units, alerts were generated but not sent, and standard RRT activation criteria were used.</p><p><strong>Measurements and main results: </strong>The primary outcome was the rate of escalation per 1000 patient bed days. Secondary outcomes included the frequency of orders for fluids, medications, and diagnostic tests, and combined in-hospital and 30-day mortality. Propensity score modeling with stabilized inverse probability of treatment weight (IPTW) was used to account for differences between groups. Data from 2740 patients enrolled between July 2019 and March 2020 were analyzed (1488 intervention, 1252 control). Average age was 66.3 years and 1428 participants (52%) were female. The rate of escalation was 12.3 vs. 11.3 per 1000 patient bed days (difference, 1.0; 95% CI, -2.8 to 4.7) and IPTW adjusted incidence rate ratio 1.43 (95% CI, 1.16-1.78; p < 0.001). Patients in the intervention group were more likely to receive cardiovascular medication orders (16.1% vs. 11.3%; 4.7%; 95% CI, 2.1-7.4%) and IPTW adjusted relative risk (RR) (1.74; 95% CI, 1.39-2.18; p < 0.001). Combined in-hospital and 30-day-mortality was lower in the intervention group (7% vs. 9.3%; -2.4%; 95% CI, -4.5% to -0.2%) and IPTW adjusted RR (0.76; 95% CI, 0.58-0.99; p = 0.045).</p><p><strong>Conclusions: </strong>Real-time machine learning alerts do not reduce the rate of escalation but may reduce mortality.</p>\",\"PeriodicalId\":10765,\"journal\":{\"name\":\"Critical Care Medicine\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":7.7000,\"publicationDate\":\"2024-07-01\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Critical Care Medicine\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1097/CCM.0000000000006243\",\"RegionNum\":1,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"2024/2/21 0:00:00\",\"PubModel\":\"Epub\",\"JCR\":\"Q1\",\"JCRName\":\"CRITICAL CARE MEDICINE\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Critical Care Medicine","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1097/CCM.0000000000006243","RegionNum":1,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"2024/2/21 0:00:00","PubModel":"Epub","JCR":"Q1","JCRName":"CRITICAL CARE MEDICINE","Score":null,"Total":0}
引用次数: 0
摘要
目的:机器学习算法在预测临床病情恶化方面优于旧式方法,但有关其实际功效的严格前瞻性数据却很有限。我们假设,由机器学习生成的实时警报直接发送给一线医疗服务提供者将减少病情升级:单中心前瞻性实用非随机分组临床试验:学术性三级医疗中心:患者:四个内外科病房收治的成人患者。根据最初入院情况决定分配到干预组还是对照组:根据预测的病情恶化可能性进行分层,向初级团队或直接向快速反应团队(RRT)发送实时警报。临床护理和干预由医疗服务提供者自行决定。在对照组中,发出警报但不发送,采用标准的 RRT 启动标准:主要结果是每 1000 个病人床日的病情升级率。次要结果包括输液、用药和诊断检测指令的频率,以及住院和 30 天的综合死亡率。采用稳定反向治疗概率权重(IPTW)倾向评分模型来考虑组间差异。对2019年7月至2020年3月期间入组的2740名患者的数据进行了分析(干预组1488人,对照组1252人)。平均年龄为 66.3 岁,1428 名参与者(52%)为女性。每 1000 个患者床日的病情升级率为 12.3 vs. 11.3(差异为 1.0;95% CI,-2.8 至 4.7),IPTW 调整后的发病率比为 1.43(95% CI,1.16-1.78;p < 0.001)。干预组患者更有可能收到心血管药物医嘱(16.1% vs. 11.3%;4.7%;95% CI,2.1-7.4%),IPTW调整后的相对风险(RR)(1.74;95% CI,1.39-2.18;p < 0.001)。干预组的院内和30天综合死亡率较低(7% vs. 9.3%; -2.4%; 95% CI, -4.5% to -0.2%),IPTW调整RR(0.76; 95% CI, 0.58-0.99; p = 0.045):实时机器学习警报不会降低病情升级率,但可以降低死亡率。
Real-Time Machine Learning Alerts to Prevent Escalation of Care: A Nonrandomized Clustered Pragmatic Clinical Trial.
Objectives: Machine learning algorithms can outperform older methods in predicting clinical deterioration, but rigorous prospective data on their real-world efficacy are limited. We hypothesized that real-time machine learning generated alerts sent directly to front-line providers would reduce escalations.
Patients: Adult patients admitted to four medical-surgical units. Assignment to intervention or control arms was determined by initial unit admission.
Interventions: Real-time alerts stratified according to predicted likelihood of deterioration sent either to the primary team or directly to the rapid response team (RRT). Clinical care and interventions were at the providers' discretion. For the control units, alerts were generated but not sent, and standard RRT activation criteria were used.
Measurements and main results: The primary outcome was the rate of escalation per 1000 patient bed days. Secondary outcomes included the frequency of orders for fluids, medications, and diagnostic tests, and combined in-hospital and 30-day mortality. Propensity score modeling with stabilized inverse probability of treatment weight (IPTW) was used to account for differences between groups. Data from 2740 patients enrolled between July 2019 and March 2020 were analyzed (1488 intervention, 1252 control). Average age was 66.3 years and 1428 participants (52%) were female. The rate of escalation was 12.3 vs. 11.3 per 1000 patient bed days (difference, 1.0; 95% CI, -2.8 to 4.7) and IPTW adjusted incidence rate ratio 1.43 (95% CI, 1.16-1.78; p < 0.001). Patients in the intervention group were more likely to receive cardiovascular medication orders (16.1% vs. 11.3%; 4.7%; 95% CI, 2.1-7.4%) and IPTW adjusted relative risk (RR) (1.74; 95% CI, 1.39-2.18; p < 0.001). Combined in-hospital and 30-day-mortality was lower in the intervention group (7% vs. 9.3%; -2.4%; 95% CI, -4.5% to -0.2%) and IPTW adjusted RR (0.76; 95% CI, 0.58-0.99; p = 0.045).
Conclusions: Real-time machine learning alerts do not reduce the rate of escalation but may reduce mortality.
期刊介绍:
Critical Care Medicine is the premier peer-reviewed, scientific publication in critical care medicine. Directed to those specialists who treat patients in the ICU and CCU, including chest physicians, surgeons, pediatricians, pharmacists/pharmacologists, anesthesiologists, critical care nurses, and other healthcare professionals, Critical Care Medicine covers all aspects of acute and emergency care for the critically ill or injured patient.
Each issue presents critical care practitioners with clinical breakthroughs that lead to better patient care, the latest news on promising research, and advances in equipment and techniques.