Ten tips for successful assessment of risk of bias in randomized trials using the RoB 2 tool: Early lessons from Cochrane

Abstract

Introduction

RoB 2 is a tool used by systematic reviewers to assess risk of bias in randomized trials. Over a period of 19 months working as editors for Cochrane, we saw many instances where users of RoB 2 frequently applied the tool in ways the developers had not intended, despite availability of detailed guidance, webinars and FAQs.

Methods

In this paper we highlight the ten main issues that we observed, with the aims of optimising the application of the RoB 2 tool, avoiding some of the frequent misapplications of the tool.

Results

Issues noted included failure to state an effect of interest, applying the tool to an entire study rather than to a specific numerical result, omitting key signaling questions and relying on outdated views of causes of bias.

Conclusion

Such omissions and misapplications can lead to overly harsh or lenient assessments of bias with potential to change the confidence we have in an evidence base of randomized trials. We recommend that teams planning to use RoB 2 include at least one member familiar with the RoB 2 detailed guidance and that they use the free resources, such as webinars and FAQs, from the developers of RoB 2 and Cochrane. Our ten tips should be useful to non-Cochrane systematic reviewers as well as to peer reviewers and editors in Cochrane and other journals.