On a Holm-related MTP for rejecting at least k hypotheses: general validity, optimality property, confidence regions, and applications.

Abstract

This article concerns p-value-based multiple testing procedures (MTPs) that can be used in a confirmatory clinical study under minimal assumptions in case the requirement for study-success is that at least k out of m primary/important hypotheses become rejected. Recently, a simple, generally valid Holm-type MTP was discussed that can be used for such a requirement for any k from one to m. It can only reject at least k (or zero) hypotheses, but this increases the power to reject k or more hypotheses compared to Holm's step-down MTP. The present article provides a simple formulation and proof of strong family-wise error rate (FWER) control for a stepwise MTP that is sharper in that for any k strictly between one and m it: (a) always rejects at least as much, and (b) can potentially reject fewer than k hypotheses. This sharper MTP too is generally valid, without any assumption about logical or stochastic relationships. It has a gatekeeping step, followed by m steps where ordered primary p-values are compared to critical constants and rejections are made in a step-down manner. These constants have the optimality property that under a natural monotonicity restriction, they cannot be increased without losing the general strong FWER control. Confidence regions like those for Holm's MTP are provided. Applications are discussed in connection with three interesting approaches proposed earlier for confirmatory studies: (a) the Superiority-Noninferiority approach; (b) Fallback tests for co-primary endpoints; and (c) Multistage gatekeeping MTPs that utilize so-called k-truncated Holm MTPs in some stages.