Measuring diagnostic test performance using imperfect reference tests: A partial identification approach

Abstract

Diagnostic tests are almost never perfect. Studies quantifying their performance use knowledge of the true health status, measured with a reference diagnostic test. Researchers commonly assume that the reference test is perfect, which is often not the case in practice. When the assumption fails, conventional studies identify “apparent” performance or performance with respect to the reference, but not true performance. This paper provides the smallest possible bounds on the measures of true performance — sensitivity (true positive rate) and specificity (true negative rate), or equivalently false positive and negative rates, in standard settings. Implied bounds on policy-relevant parameters are derived: (1) Prevalence in screened populations; (2) Predictive values. Methods for inference based on moment inequalities are used to construct uniformly consistent confidence sets in level over a relevant family of data distributions. Emergency Use Authorization (EUA) and independent study data for the BinaxNOW COVID-19 antigen test demonstrate that the bounds can be very informative. Analysis reveals that the estimated false negative rates for symptomatic and asymptomatic patients are up to 3.17 and 4.59 times higher than the frequently cited “apparent” false negative rate. Further applicability of the results in the context of imperfect proxies such as survey responses and imputed protected classes is indicated.