Lack of reproducibility of trial sequential analyses: a meta-epidemiological study.

Abstract

Systematic reviews and meta-analyses are essential tools for synthesizing evidence from multiple studies. Recently, trial sequential analyses (TSAs) have gained popularity as a component of meta-analyses, helping researchers dynamically monitor evidence as new studies are incorporated. This article introduces a meta-epidemiological study aimed at evaluating the reproducibility of TSAs within systematic reviews published in 2023. Two independent investigators assessed and reproduced the main TSA for each included systematic review. Our search in PubMed yielded a convenience sample of 98 systematic reviews. Only 28% (27/98) of the included TSAs provided sufficient data to calculate the required information size, an essential element for assessing statistical power and conducting TSAs. Among these, 81% (22/27) provided the necessary data to determine decision boundaries and Z-curves in TSAs. Overall, full reproducibility was achieved for only 13% (13/98) of TSAs. Specifically, for binary outcomes, 65% (47/72) of TSAs failed to report event rates in control groups, and 44% (32/72) did not report relative risk reductions. For continuous outcomes, 53% (17/32) failed to report minimally relevant differences, and 72% (23/32) did not report variances. These elements are crucial for TSA reproducibility. Moreover, the reproducibility of TSAs was associated with journal impact factors and adherence to the PRISMA guidelines. A collective effort is needed from systematic review authors, peer reviewers, and journal editors to improve the reproducibility of TSAs.