Variability in Serious Safety Event Classification among Children’s Hospitals: A Measure for Comparison?

Introduction: Hospitals have no standard for measuring comparative rates of serious safety events (SSE). A pediatric hospital safety collaborative has used a common definition and measurement system to classify SSE and calculate a serious safety event rate. An opportunity exists to evaluate the use of this measurement system. Methods: A web-based survey utilizing 7 case vignettes was sent to 132 network hospitals to assess agreement in classifying the vignettes as SSEs. Respondents classified the vignettes according to the taxonomy used at their respective organizations for deviations and SSE classification. Results: Of the 82 respondents, 67 (82%) utilized the same SSE classification system. Respondents did not assess deviations for 2 of the 7 vignettes, which had clear deviations. Of the remaining 5 vignettes, 3 had a substantial agreement of deviation (>85%, Gwet’s AC ≥ 0.68), and 2 had fair agreement (<70%, Gwet’s AC ≤ 0.39). Four of the 7 vignettes had a substantial agreement on SSE classification (>80%; Gwet’s AC ≥ 0.80), and 3 had slight to moderate agreement (<70%, Gwet’s AC ≤ 0.78). Conclusions: Results demonstrated agreement and variability in determining deviation and SSE classification in the 7 vignettes. Although the SSE methodology and metric used by participant pediatric hospitals yields generally similar review results, one must be cautious in using the SSE rate to compare patient safety outcomes across different hospitals.