Confidence Interval Comparisons For Probability of Detection On Hit/Miss Data

Probability of detection (POD) studies for evaluating the capabilities of an inspection system for Air Force aircraft structural components commonly use a Logistic Regression model with a Wald 95% confidence interval. However, hit/miss POD data is distributed as a Binomial, and the sample sizes are commonly too small for Wald’s identically and independently normality distributed assumption to be true. This paper uses a large set of simulated representative hit/miss data to compare and contrast the performance of the four confidence intervals methods: Standard Wald, Modified Wald, Profile Likelihood Ratio, and Profile Modified Likelihood Ratio. Performance is measured in terms of bias and existence of a90/95 with respect to data distribution, sample size, overlap, and evenness. This paper provides guidance and methodology on new POD methods that more reliably and accurately estimate a90/95.