To adjust, or not to adjust, for multiple comparisons

Abstract

Questions often arise concerning when, whether, and how we should adjust our interpretation of the results from multiple hypothesis tests. Strong arguments have been put forward in the epidemiological literature against any correction or adjustment for multiplicity, but regulatory requirements (particularly for pharmaceutical trials) can sometimes trump other concerns. The formal basis for adjustment is often the control of error rates, and hence the problems of multiplicity may seem rooted in a purely frequentist paradigm, though this can be a restrictive viewpoint. Commentators may never wholly agree on any of these things. This article draws some of the key threads from the discussion and suggests further reading.