Comparison of risk-adjusted cumulative quality control charts compared with standardized mortality ratios in critical care.

IF 3.4 3区医学 Q1 ANESTHESIOLOGY Canadian Journal of Anesthesia-Journal Canadien D Anesthesie Pub Date : 2024-11-07 DOI:10.1007/s12630-024-02863-6

Claudio M Martin, Fran Priestap, Raymond Kao

{"title":"Comparison of risk-adjusted cumulative quality control charts compared with standardized mortality ratios in critical care.","authors":"Claudio M Martin, Fran Priestap, Raymond Kao","doi":"10.1007/s12630-024-02863-6","DOIUrl":null,"url":null,"abstract":"Purpose: The optimal method for monitoring intensive care unit (ICU) performance is unknown. We sought to compare process control charts using standardized mortality ratio (SMR), p-charts, and cumulative sum (CUSUM) charts for detecting increases in risk-adjusted mortality within ICUs.Methods: Using data from 17 medical-surgical ICUs that included 29,592 patients in Ontario, Canada, we created risk-adjusted p-charts and SMRs on monthly intervals and CUSUM charts. We defined positive signals as any data point that was above the 3-sigma limit (approximating a 99% confidence interval [CI]) on a p-chart, any data point whose 95% CI did not include 1 for the SMR charts, and when a data point exceeded control limits for an odds ratio of 1.5 for CUSUM charts. We simulated increases in mortality of 10%, 30%, and 50% for each ICU to determine the sensitivity of each method. We calculated sensitivity as the number of positive signals divided by the number of ICUs (equal to number of simulated events).Results: Cumulative sum charts generated 31 signals in 12 different ICUs, while p-charts and SMR agreed in 10 and 6 of these signals, respectively, followed by 21 signals from p-charts across 14 ICUs (agreement in 10 of these signals for both CUSUM and SMR) and 15 signals from SMR charts across eight ICUs (agreement from p-charts and CUSUM in 10 and six signals, respectively). The p-chart had a sensitivity of 88% (95% CI, 73 to 104) for a 50% simulated increase in ICU mortality followed by CUSUM at 71% (95% CI, 49 to 102) and SMR at 59% (95% CI, 35 to 82). Performance with lower simulated increases was poor for all three methods.Conclusions: P-charts created with risk-adjusted mortality at monthly intervals are potentially useful tools for monitoring ICU performance. Future studies should consider usability testing with ICU leaders and application of these methods to additional clinical domains.","PeriodicalId":56145,"journal":{"name":"Canadian Journal of Anesthesia-Journal Canadien D Anesthesie","volume":null,"pages":null},"PeriodicalIF":3.4000,"publicationDate":"2024-11-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Canadian Journal of Anesthesia-Journal Canadien D Anesthesie","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1007/s12630-024-02863-6","RegionNum":3,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ANESTHESIOLOGY","Score":null,"Total":0}

引用次数: 0

Abstract

Purpose: The optimal method for monitoring intensive care unit (ICU) performance is unknown. We sought to compare process control charts using standardized mortality ratio (SMR), p-charts, and cumulative sum (CUSUM) charts for detecting increases in risk-adjusted mortality within ICUs.

Methods: Using data from 17 medical-surgical ICUs that included 29,592 patients in Ontario, Canada, we created risk-adjusted p-charts and SMRs on monthly intervals and CUSUM charts. We defined positive signals as any data point that was above the 3-sigma limit (approximating a 99% confidence interval [CI]) on a p-chart, any data point whose 95% CI did not include 1 for the SMR charts, and when a data point exceeded control limits for an odds ratio of 1.5 for CUSUM charts. We simulated increases in mortality of 10%, 30%, and 50% for each ICU to determine the sensitivity of each method. We calculated sensitivity as the number of positive signals divided by the number of ICUs (equal to number of simulated events).

Results: Cumulative sum charts generated 31 signals in 12 different ICUs, while p-charts and SMR agreed in 10 and 6 of these signals, respectively, followed by 21 signals from p-charts across 14 ICUs (agreement in 10 of these signals for both CUSUM and SMR) and 15 signals from SMR charts across eight ICUs (agreement from p-charts and CUSUM in 10 and six signals, respectively). The p-chart had a sensitivity of 88% (95% CI, 73 to 104) for a 50% simulated increase in ICU mortality followed by CUSUM at 71% (95% CI, 49 to 102) and SMR at 59% (95% CI, 35 to 82). Performance with lower simulated increases was poor for all three methods.

Conclusions: P-charts created with risk-adjusted mortality at monthly intervals are potentially useful tools for monitoring ICU performance. Future studies should consider usability testing with ICU leaders and application of these methods to additional clinical domains.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

重症监护中风险调整累积质量控制图与标准化死亡率的比较。

目的：监测重症监护病房（ICU）绩效的最佳方法尚不清楚。我们试图比较使用标准化死亡率（SMR）、p 图表和累积总和（CUSUM）图表的流程控制图，以检测 ICU 内风险调整后死亡率的增加情况：我们利用加拿大安大略省 17 个内外科重症监护病房（包括 29,592 名患者）的数据，创建了风险调整后的 p 图表、月间隔 SMR 和 CUSUM 图表。我们将积极信号定义为：在 p 图表中，任何数据点超过 3 西格玛限制（近似于 99% 置信区间 [CI]）；在 SMR 图表中，任何数据点的 95% CI 不包括 1；在 CUSUM 图表中，当一个数据点超过控制限制时，几率比为 1.5。我们模拟每个 ICU 的死亡率分别增加 10%、30% 和 50%，以确定每种方法的灵敏度。我们用阳性信号的数量除以 ICU 的数量（等于模拟事件的数量）来计算灵敏度：累积总和图表在 12 个不同的 ICU 中产生了 31 个信号，而 p 图表和 SMR 分别在其中的 10 个和 6 个信号中达成一致，其次是 p 图表在 14 个 ICU 中产生的 21 个信号（CUSUM 和 SMR 在其中的 10 个信号中达成一致），以及 SMR 图表在 8 个 ICU 中产生的 15 个信号（p 图表和 CUSUM 分别在其中的 10 个和 6 个信号中达成一致）。当 ICU 死亡率模拟上升 50%时，p-图表的灵敏度为 88%（95% CI，73-104），CUSUM 为 71%（95% CI，49-102），SMR 为 59%（95% CI，35-82）。所有三种方法在模拟死亡率增加较低时的表现都很差：结论：以每月为间隔用风险调整死亡率绘制的 P 图表是监测重症监护室绩效的潜在有用工具。未来的研究应考虑对 ICU 领导进行可用性测试，并将这些方法应用到其他临床领域。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Canadian Journal of Anesthesia-Journal Canadien D Anesthesie 医学-麻醉学

CiteScore

8.50

自引率

7.10%

发文量

161

审稿时长

6-12 weeks

期刊介绍： The Canadian Journal of Anesthesia (the Journal) is owned by the Canadian Anesthesiologists’ Society and is published by Springer Science + Business Media, LLM (New York). From the first year of publication in 1954, the international exposure of the Journal has broadened considerably, with articles now received from over 50 countries. The Journal is published monthly, and has an impact Factor (mean journal citation frequency) of 2.127 (in 2012). Article types consist of invited editorials, reports of original investigations (clinical and basic sciences articles), case reports/case series, review articles, systematic reviews, accredited continuing professional development (CPD) modules, and Letters to the Editor. The editorial content, according to the mission statement, spans the fields of anesthesia, acute and chronic pain, perioperative medicine and critical care. In addition, the Journal publishes practice guidelines and standards articles relevant to clinicians. Articles are published either in English or in French, according to the language of submission.