Within-Laboratory Variance Outlier Detection: An Alternative to Cochran’s Test

Abstract

Summary An important step in the full definition of an analytical method is the characterization of the within and between laboratory variability. This is typically done through collaborative studies involving multiple laboratories. The statistical analysis of the results of collaborative studies is generally carried out using standardized protocols such as those given in ISO 5725-2 or ASTM E691-14. One aspect of the evaluation of collaborative studies is the identification of outlying laboratories which are then excluded from the variance calculation associated with the analytical method. Whether particular laboratories are identified as outliers can have a dramatic effect on the calculated variances. The generally recommended approach to identify laboratories with excessive within-laboratory variation is Cochran’s Test or something similar. However, Cochran’s Test is very sensitive to non-normality of the underlying statistical distribution. When the assumption of normality is violated, Cochran’s Test can wrongly identify laboratories as outliers at much greater than the nominally stated error rate, even for deviations from normality that are very difficult to detect analytically. In this paper, an alternative to Cochran’s Test, adapted from Levene’s Test, is proposed and shown to approximately maintain the stated error rate when the underlying distribution is not normal. This newly adapted test is recommended for future collaborative study analysis in place of Cochran’s Test.