Beyond the Fragility Index.

Abstract

The results of randomized clinical trials (RCTs) are frequently assessed with the fragility index (FI). Although the information provided by FI may supplement the p value, this indicator presents intrinsic weaknesses and shortcomings. In this article, we establish an analysis of fragility within a broader framework so that it can reliably complement the information provided by the p value. This perspective is named the analysis of strength. We first propose a new strength index (SI), which can be adopted in normal distribution settings. This measure can be obtained for both significance and nonsignificance and is straightforward to calculate, thus presenting compelling advantages over FI, starting from the presence of a threshold. The case of time-to-event outcomes is also addressed. Then, beyond the p value, we develop the analysis of strength using likelihood ratios from Royall's statistical evidence viewpoint. A new R package is provided for performing strength calculations, and a simulation study is conducted to explore the behavior of SI and the likelihood-based indicator empirically across different settings. The newly proposed analysis of strength is applied in the assessment of the results of three recent trials involving the treatment of COVID-19.