Let's practice what we preach: Planning and interpreting simulation studies with design and analysis of experiments

Abstract

Statisticians recommend design and analysis of experiments (DAE) for evidence-based research but often use tables to present their own simulation studies. Could DAE do better? We outline how DAE methods can be used to plan and analyze simulation studies. Tools for planning include cause-and-effect diagrams and factorial and fractional factorial designs. Analysis is carried out via analysis of variance, main effect and interaction plots, and other DAE tools. We also demonstrate how Taguchi robust parameter design can be used to study the robustness of methods to a variety of uncontrollable population parameters.