Validating Health Economic Models With the Probabilistic Analysis Check dashBOARD

Abstract

Objectives

Health economic (HE) models are often considered as “black boxes” because they are not publicly available and lack transparency, which prevents independent scrutiny of HE models. Additionally, validation efforts and validation status of HE models are not systematically reported. Methods to validate HE models in absence of their full underlying code are therefore urgently needed to improve health policy making.

This study aimed to develop and test a generic dashboard to systematically explore the workings of HE models and validate their model parameters and outcomes.

Methods

The Probabilistic Analysis Check dashBOARD (PACBOARD) was developed using insights from literature, health economists, and a data scientist.

Functionalities of PACBOARD are (1) exploring and validating model parameters and outcomes using standardized validation tests and interactive plots, (2) visualizing and investigating the relationship between model parameters and outcomes using metamodeling, and (3) predicting HE outcomes using the fitted metamodel.

To test PACBOARD, 2 mock HE models were developed, and errors were introduced in these models, eg, negative costs inputs, utility values exceeding 1. PACBOARD metamodeling predictions of incremental net monetary benefit were validated against the original model’s outcomes.

Results

PACBOARD automatically identified all errors introduced in the erroneous HE models. Metamodel predictions were accurate compared with the original model outcomes.

Conclusions

PACBOARD is a unique dashboard aiming at improving the feasibility and transparency of validation efforts of HE models. PACBOARD allows users to explore the working of HE models using metamodeling based on HE models’ parameters and outcomes.