Development and validation of a machine learning–based, point-of-care risk calculator for post-ERCP pancreatitis and prophylaxis selection

Abstract

Background and Aims

A robust model of post-ERCP pancreatitis (PEP) risk is not currently available. We aimed to develop a machine learning–based tool for PEP risk prediction to aid in clinical decision making related to periprocedural prophylaxis selection and postprocedural monitoring.

Methods

Feature selection, model training, and validation were performed using patient-level data from 12 randomized controlled trials. A gradient-boosted machine (GBM) model was trained to estimate PEP risk, and the performance of the resulting model was evaluated using the area under the receiver operating curve (AUC) with 5-fold cross-validation. A web-based clinical decision-making tool was created, and a prospective pilot study was performed using data from ERCPs performed at the Johns Hopkins Hospital over a 1-month period.

Results

A total of 7389 patients were included in the GBM with an 8.6% rate of PEP. The model was trained on 20 PEP risk factors and 5 prophylactic interventions (rectal nonsteroidal anti-inflammatory drugs [NSAIDs], aggressive hydration, combined rectal NSAIDs and aggressive hydration, pancreatic duct stenting, and combined rectal NSAIDs and pancreatic duct stenting). The resulting GBM model had an AUC of 0.70 (65% specificity, 65% sensitivity, 95% negative predictive value, and 15% positive predictive value). A total of 135 patients were included in the prospective pilot study, resulting in an AUC of 0.74.

Conclusions

This study demonstrates the feasibility and utility of a novel machine learning–based PEP risk estimation tool with high negative predictive value to aid in prophylaxis selection and identify patients at low risk who may not require extended postprocedure monitoring.