Discrete event simulation for emergency department modelling: A systematic review of validation methods

Background:

Discrete event simulation has been widely used for decades to model the Emergency Department (ED) patient flow, understand system bottlenecks, and analyse resource capacity planning constraints. Given the complexity and constraints related to modelling the ED setting, such as data availability, there is often a gap in prescribing the correct validation approach.

Objectives:

The purpose of this review study is to come up with practical guidelines for employing specific validation techniques by comparing the available “best practice” simulation validation approaches against the approaches commonly found in published ED Discrete Event Simulation (DES) studies.

Methods:

We conducted a systematic review of the peer-reviewed literature to identify DES studies of patient flow within hospital EDs across the globe. Our search strategy focused on two main domains of knowledge associated with the current study: DES and ED. In total, we selected 90 studies a basis for the analysis. Additionally, we have identified a total of 7 papers focused on best practice approaches in validation.

Results:

A plurality of studies only discuss a single type of validation (data-led) at 30%, closely followed by none (23%), and data-led paired with qualitative validation, but no verification (at 22%). LOS is the most common validation metric, with 47% of studies that used data-led validation selecting length of stay (LOS) as the key validation metric. The next most frequently validated metric is throughput (9%), followed by time to triage (TTT) (8%). “Confidence Interval” and “% Difference” are by far the most common, with 32% of studies employing the former, and 22% employing the latter. Remaining techniques tend to be used more sporadically, with hypothesis testing, correlation coefficient, Student t-test, and Welch’s two-sample t-test being the most frequent (5% to 9% of studies).

Conclusion:

Based on the reviewed studies, we propose guidelines for the validation procedure given five “levels” of available data quality. The guideline incorporates both the analysis of best practice literature, as well as the trends for validation based on the review of 90 generic and specific simulation studies.