Characterization of a credibility index.

Abstract

In recent years, the role of the p-value in applied research has been heavily scrutinized. Several new proposals have been put forward from a Bayesian viewpoint, including the analysis of credibility. By using the reverse Bayes theorem, and reasoning in terms of subverting the significance or the non-significance denoted by the p-value, this analysis provides the credibility, in a Bayesian sense, of an experimental result. We discuss a normalized indicator of credibility, namely $C$, a variant of the index $\tilde{C}$ (Quatto et al. J. Biopharm. Stat. 32, 308-329, 2022). This can be used to assess the degree of credibility of experimental results and can also be compared with a fixed threshold. The index is extended to the case of one-sided hypotheses. A simulation study is conducted to empirically assess the behavior of the index $C$. Two illustrative examples in the contexts of pharmacotherapy for COVID-19 and heart failure are presented. We then propose adopting the credibility index for meta-analyses, in which it can provide a suitable diagnostic value for modeling fixed and random effects.