Statistical Tests of Type I Error

Abstract

This chapter covers tests of statistical significance that can be used to compare data across phases. These are used to determine whether observed outcomes are likely the result of an intervention or, more likely, the result of sampling error or chance. The purpose of a statistical test is to determine how likely it is that the analyst is making an incorrect decision by rejecting the null hypothesis, that there is no difference between compared phases, and accepting the alternative one, that true differences exist. A number of tests of significance are presented in this chapter: statistical process control charts (SPCs), proportion/frequency, chi-square, the conservative dual criteria (CDC), robust conservative dual criteria (RCDC), the t test, and analysis of variance (ANOVA). How and when to use each of these are also discussed, and examples are provided to illustrate each. The method for transforming autocorrelated data and merging data sets is discussed further in the context of utilizing transformed data sets to test of Type 1 error.