Implementation and prospective performance evaluation of an intraoperative duration prediction model using high throughput real-time data

Background

Accurate real-time prediction of intraoperative duration can contribute to improved perioperative outcomes. We implemented a data pipeline for extraction of real-time data from nascent anaesthesia records and silently deployed a predictive machine learning (ML) algorithm.

Methods

Clinical variables were retrieved from the electronic health record via a third-party clinical decision support platform and contemporaneously ingested into a previously developed ML model. The model was trained using 3 months data, and performance was subsequently evaluated over 10 months using continuous ranked probability score.

Results

The ML model made 6 173 435 predictions on 62 142 procedures. Mean continuous ranked probability score for the ML model was 27.19 (standard error 0.016) min compared with 51.66 (standard error 0.029) min for the bias-corrected scheduled duration. Linear regression did not demonstrate performance drift over the testing period.

Conclusions

We implemented and silently deployed a real-time ML algorithm for predicting surgery duration. Prospective evaluation showed that model performance was preserved over a 10-month testing period.