On the generation and ownership of alpha in medical studies

Abstract

Much is known about how to split alpha between or among several comparisons, or how to preserve the nominal alpha level with an exact analysis, but the issue of how alpha is generated, or where it comes from, has not received a commensurate degree of attention. It would seem that there is little point in working out methods to allocate or conserve alpha if it is unlimited in supply. Moreover, there seems to be a logical inconsistency in requiring that a given amount of alpha, generally 0.05, be split among the primary comparisons performed by a given set of researchers, yet allowing other researchers to analyze the same data with a new 0.05 to work with. We will address these inconsistencies, and ask more generally where alpha comes from, how it can be generated, and under what conditions it should be one-tailed or two-tailed.