The Fragility Index Minimally Improves Interpretation of the Medical Literature: A Boat Made of Bricks in a Sea of Uncertainty.

Abstract

The fragility index (FI) is statistical significance in a costume. Perhaps attractive and amusing, but behind the mask it's nothing more than spin, dichotomizing results as "statistically significant" versus "not". In the medical literature, we must stop dichotomizing and start measuring the magnitude of effect and the uncertainty in this estimate. Statistical significance is thought stifling. Yet, it is the tool with which the medical research community has been provided. No wonder we dichotomize results; we've been encouraged to do so. The question is, "Will we recognize the folly in this exercise and move on to more critical questions of relevance and accuracy of published research?" The FI is heralded as a metric that provides insight beyond statistical significance. Rather than provide a measure of uncertainty, which is what fragility implies, it quantifies the number of patients needed to produce a p-value that's greater than 0.05. Unfortunately, while well intended the FI is not a surrogate for robustness of clinical trial data, nor the underlying statistical analysis. In contrast, reporting and interpreting a confidence interval more effectively provides a sense of uncertainty. While far from perfect, the confidence interval provides a range of values that are compatible with the observed study data. This makes the uncertainty of the data transparent. Advancing our understanding of the data starts with stepping away from statistical significance.