Multiplying Alpha: When Statistical Tests Compound in Sports Medicine Research.

Abstract

Scientific inquiry aims to minimize bias and ensure accurate conclusions. A critical yet often overlooked issue in sports medicine and exercise science research is the family-wise error rate (FWER) and experimental-wise error rate (EWER), which increase with multiple statisticalinferences, inflating the risk of Type I errors. While FWER corrections are standard in post-hoc ANOVA tests, they are inconsistently applied in broader research contexts. Using an example from our research team of over 67 million regression models, we illustrate how failing to correct for FWER can create spurious findings. Approximately 3 million (4.4%) models werestatistically significant (p<0.05), aligning with the expected false-positive rate. This underscores the necessity of solutions such as preregistration, false discovery rate control, and Bayesian approaches. Without proper corrections, erroneous conclusions may mislead clinical decision- making and potentially harm patients, highlighting the importance of rigorous statistical methods in evidence-based sports medicine.