The Umbrella Collaboration®: An Innovative Tertiary Evidence Synthesis Methodology. Validation Study Protocol.

Abstract

Background: The synthesis of evidence in healthcare is essential for informed decision-making and policy development. This study aims to validate The Umbrella Collaboration® (TU®), an innovative, semi-automatic tertiary evidence synthesis methodology, by comparing it with Traditional Umbrella Reviews (TUR), which are currently the gold standard.

Objective: The primary objective of this study is to evaluate whether TU®, an AI-assisted, software-driven system for tertiary evidence synthesis, can achieve comparable effectiveness to TURs, while offering a more timely, efficient, and comprehensive approach. Additionally, as a secondary objective, the study aims to assess the accessibility and comprehensibility of TU®'s outputs to ensure its usability and practical applicability for healthcare professionals.

Methods: This protocol outlines a comparative study divided into two main parts. The first part involves a quantitative comparison of results obtained using TU® and TURs in geriatrics. We will evaluate the identification, size effect, direction, statistical significance, and certainty of outcomes, as well as the time and resources required for each methodology. Data for TURs will be sourced from Medline (via PubMed), while TU® will use AI-assisted informatics to replicate the research questions of the selected URTs. The second part of the study assesses the ease of use and comprehension of TU® through an online survey directed at health professionals, utilizing interactive features and detailed data access.

Results: Expected results include the assessment of concordance in identifying outcomes, the size effect, direction and significance of these outcomes, and the certainty of evidence. Additionally, we will measure the operational efficiency of each methodology by evaluating the time taken to complete projects. User perceptions of the ease of use and comprehension of TU® will be gathered through detailed surveys. The implementation of new methodologies in evidence synthesis requires validation. This study will determine whether TU® can match the accuracy and comprehensiveness of TURs while offering benefits in terms of efficiency and user accessibility. The comparative study is designed to address the inherent challenges in validating a new methodology against established standards.

Conclusions: If TU® proves as effective as TURs but more time-efficient, accessible and easily updatable, it could significantly enhance the process of evidence synthesis, facilitating informed decision-making and improving healthcare. This study represents a step towards integrating innovative technologies into routine evidence synthesis practice, potentially transforming health research.

Clinicaltrial: