Adding imprecision to hypotheses: A Bayesian framework for testing practical significance in nonparametric settings

Abstract

Instead of testing solely a precise hypothesis, it is often useful to enlarge it with alternatives deemed to differ negligibly from it. For instance, in a bioequivalence study one might test if the concentration of an ingredient is exactly the same in two drugs. In such a context, it might be more relevant to test the enlarged hypothesis that the difference in concentration between them is of no practical significance. While this concept is not alien to Bayesian statistics, applications remain mostly confined to parametric settings and strategies that effectively harness experts' intuitions are often scarce or nonexistent. To resolve both issues, we introduce the Pragmatic Region Oriented Test (PROTEST), an accessible nonparametric testing framework based on distortion models that can seamlessly integrate with Markov Chain Monte Carlo (MCMC) methods and is available as an R package. We develop expanded versions of model adherence, goodness-of-fit, quantile and two-sample tests. To demonstrate how PROTEST operates, we use examples, simulated studies that critically evaluate features of the test and an application on neuron spikes. Furthermore, we address the crucial issue of selecting the threshold—which controls how much a hypothesis is to be expanded—even when intuitions are limited or challenging to quantify.