Trustworthiness assessment as an inclusion criterion for systematic reviews—What is the impact on results?

Abstract

Background

There is increasing concern that a significant proportion of randomized controlled trials (RCTs) included in Cochrane reviews may not be trustworthy. Applying the Cochrane Pregnancy and Childbirth Trustworthiness Screening Tool (CPC-TST) has already had a clinically important effect on several reviews published by the Cochrane Pregnancy and Childbirth Group.

Objectives

We wanted to assess the impact of removing untrustworthy RCTs from already-published Cochrane reviews on a defined clinical area (antenatal and postnatal nutritional interventions).

Methods

We applied the tool to 18 Cochrane reviews (374 RCTs). The tool had four domains: (i) is the research governance trustworthy; (ii) are the baseline characteristics trustworthy; (iii) is the study feasible; (iv) are the results plausible? When additional information was needed, authors were contacted using a standard template. At least two attempts were made to contact the authors. At the end of the evaluation process each study was classified as: (i) included (YES to all questions); (ii) excluded (retracted study); or (iii) awaiting classification (any NO to the questions).

Results

Ninety-three out of 374 included studies (25%) were reclassified as “excluded” or “awaiting classification.” The number of included RCTs was reduced in 14 out of 18 reviews. Six reviews (33%) were judged to require updating because of important differences in the Summary of Findings tables (direction and size of effects and/or GRADE ratings), conclusions, implication for practice, and/or implication for research.

Conclusions

Formal assessment of trustworthiness, and inclusion only of studies that satisfy prespecified criteria for trustworthiness, affect conclusions in a relatively large number of Cochrane reviews, with potentially important clinical implications for practice and research.