Presenting predictions and performance of probabilistic models for clinical decision support in trauma care

Abstract

Introduction

Both predictions and performance of clinical predictive models can be presented with various verbal and visual representations. This study aims to investigate how different risk and performance presentations for probabilistic predictions affect clinical users’ judgement and preferences.

Methods

We use a clinical Bayesian Network (BN) model that has been developed for predicting the risk of Trauma Induced Coagulopathy (TIC). Three patient scenarios with different levels of TIC risk were shown to trauma care clinicians. The prediction and discriminatory performance of TIC BN were shown with each scenario using different charts in a random order. Bar charts, icon arrays and gauge charts were used for presenting the prediction. Receiver operating characteristic curves, true and false positive rate curves and icon arrays were used for presenting the performance. Risk judgement for patient scenarios, perceived accuracy for the predictions and the model, and preferences for charts were elicited using an online survey.

Results

A total of 25 clinicians evaluated 75 BN predictions. The choice of risk charts was associated with the risk score in the borderline medium-risk scenario. The choice of risk and performance charts interacts with the perceived accuracy of the predictions and model in the high and low-risk scenarios, respectively. The participants had varying but persistent preferences regarding risk presentation charts. Icon arrays were preferred for performance presentations.

Conclusions

The choice of presenting predictions and the performance of predictive models can affect risk and performance interpretation. Clinical predictive models should offer the flexibility of presenting predictions with different illustrations.