Daniel Gomon, Julie Sijmons, Hein Putter, Jan Willem Dekker, Rob Tollenaar, Michel Wouters, Pieter Tanis, Marta Fiocco, Mirko Signorelli
{"title":"Inspecting the quality of care: a comparison of CUSUM methods for inter hospital performance","authors":"Daniel Gomon, Julie Sijmons, Hein Putter, Jan Willem Dekker, Rob Tollenaar, Michel Wouters, Pieter Tanis, Marta Fiocco, Mirko Signorelli","doi":"10.1007/s10742-023-00315-0","DOIUrl":null,"url":null,"abstract":"Abstract During the past 14 years, a clinical audit has been used in the Netherlands to provide hospitals with data on their performance in colorectal cancer care. Continuous feedback on the quality of care provided at each hospital is essential to improve patient outcomes. It is unclear which methods should be used to generate most informative output for the identification of potential quality issues. Our aim is to compare the commonly employed funnel plot with existing cumulative sum (CUSUM) methodology for the evaluation of postoperative survival and hospital stay outcomes of patients who underwent colorectal surgery in the Netherlands. Data from the Dutch ColoRectal Audit on 25367 patients in the Netherlands who underwent surgical resection for colorectal cancer in 71 hospitals between 2019 and 2021 is used to compare four methods for the detection of deviations in the quality of care. Two methods based on binary outcomes (funnel plot, binary CUSUM) and two CUSUM charts based on survival outcomes (BK-CUSUM and CGR-CUSUM) are considered. A novel approach for determining hospital specific control limits for CUSUM charts is proposed. The ability to detect deviations as well as the time until detection are compared for the four methods. Charts were constructed for the inspection of both postoperative survival and hospital stay. Methods using survival outcomes always yielded faster detection times compared to approaches employing binary outcomes. Detections between methods mostly coincided for postoperative survival. For hospital stay detections varied strongly, with methods based on survival outcomes signalling over half the hospitals. Further pros and cons as well as pitfalls of all methods under consideration are discussed. Methodology for the continuous inspection of the quality of care should be tailored to the specific outcome. Properly understanding how the mechanism of a control chart functions is crucial for the correct interpretation of results. This is particularly true for CUSUM charts, which require the choice of a parameter that greatly influences the results. When applying CUSUM charts, consideration of these issues is strongly recommended.","PeriodicalId":45600,"journal":{"name":"Health Services and Outcomes Research Methodology","volume":null,"pages":null},"PeriodicalIF":1.6000,"publicationDate":"2023-10-24","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Health Services and Outcomes Research Methodology","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1007/s10742-023-00315-0","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"HEALTH CARE SCIENCES & SERVICES","Score":null,"Total":0}
引用次数: 0
Abstract
Abstract During the past 14 years, a clinical audit has been used in the Netherlands to provide hospitals with data on their performance in colorectal cancer care. Continuous feedback on the quality of care provided at each hospital is essential to improve patient outcomes. It is unclear which methods should be used to generate most informative output for the identification of potential quality issues. Our aim is to compare the commonly employed funnel plot with existing cumulative sum (CUSUM) methodology for the evaluation of postoperative survival and hospital stay outcomes of patients who underwent colorectal surgery in the Netherlands. Data from the Dutch ColoRectal Audit on 25367 patients in the Netherlands who underwent surgical resection for colorectal cancer in 71 hospitals between 2019 and 2021 is used to compare four methods for the detection of deviations in the quality of care. Two methods based on binary outcomes (funnel plot, binary CUSUM) and two CUSUM charts based on survival outcomes (BK-CUSUM and CGR-CUSUM) are considered. A novel approach for determining hospital specific control limits for CUSUM charts is proposed. The ability to detect deviations as well as the time until detection are compared for the four methods. Charts were constructed for the inspection of both postoperative survival and hospital stay. Methods using survival outcomes always yielded faster detection times compared to approaches employing binary outcomes. Detections between methods mostly coincided for postoperative survival. For hospital stay detections varied strongly, with methods based on survival outcomes signalling over half the hospitals. Further pros and cons as well as pitfalls of all methods under consideration are discussed. Methodology for the continuous inspection of the quality of care should be tailored to the specific outcome. Properly understanding how the mechanism of a control chart functions is crucial for the correct interpretation of results. This is particularly true for CUSUM charts, which require the choice of a parameter that greatly influences the results. When applying CUSUM charts, consideration of these issues is strongly recommended.
期刊介绍:
The journal reflects the multidisciplinary nature of the field of health services and outcomes research. It addresses the needs of multiple, interlocking communities, including methodologists in statistics, econometrics, social and behavioral sciences; designers and analysts of health policy and health services research projects; and health care providers and policy makers who need to properly understand and evaluate the results of published research. The journal strives to enhance the level of methodologic rigor in health services and outcomes research and contributes to the development of methodologic standards in the field. In pursuing its main objective, the journal also provides a meeting ground for researchers from a number of traditional disciplines and fosters the development of new quantitative, qualitative, and mixed methods by statisticians, econometricians, health services researchers, and methodologists in other fields. Health Services and Outcomes Research Methodology publishes: Research papers on quantitative, qualitative, and mixed methods; Case Studies describing applications of quantitative and qualitative methodology in health services and outcomes research; Review Articles synthesizing and popularizing methodologic developments; Tutorials; Articles on computational issues and software reviews; Book reviews; and Notices. Special issues will be devoted to papers presented at important workshops and conferences.