Yet another problem with systematic reviews: A living review update.

Abstract

In February 2023, the Journal of Clinical Epidemiology published 'The Problems with Systematic Reviews: A living Systematic Review.' In updating this living review for the first time to incorporate literature from May 2022 to May 2023, a new problem and several themes have emerged from 152 newly included articles relating to research culture This brings the total number of relevant articles up to 637 and the total number of problems with systematic reviews up to 68. This update documents a new problem: the lack of gender diversity of systematic review author teams. It also reveals emerging themes such as: fast science from systematic reviews on COVID-19; the failure of citation of methodological or reporting guidelines to predict high-quality methodological or reporting quality; and the influence of vested interests on systematic review conclusions. These findings coupled with a proliferation of research waste from "me-too" meta-research articles highlighting well-established problems in systematic reviews underscores the need for reforms in research culture to address the incentives for producing and publishing research papers. This update reports where the identified flaws in systematic reviews affect their conclusions drawing on 77 meta-epidemiological studies from the total 637 included articles. These meta-meta-analytic studies begin the important work of examining which problems threaten the reliability and validity of treatment effects or conclusions derived from systematic reviews. We recommend that meta-research endeavours evolve from merely documenting well-established issues to understanding lesser-known problems or consequences to systematic reviews.