Discrepancies in reported results between trial registries and journal articles for AI clinical research.

Abstract

Background: Complete and unbiased reporting of clinical trial results is essential for evaluating medical advances, yet publication bias and reporting discrepancies in research on the clinical application of artificial intelligence (AI) remain unknown.

Methods: We conducted a comprehensive search of research publications and clinical trial registries focused on the application of AI in healthcare. Our search included publications in Dimensions.ai and pre-registered records from ClinicalTrials.gov and the EU Clinical Trials Registry before 31 December 2023. We linked registered trials to their corresponding publications, analysed the registration, reporting and different dissemination patterns of results, identified discrepancies between clinical trial registries and published literature, and assessed the use of these results in secondary research.

Findings: We identified 28,248 publications related to the use of AI in clinical settings and found 1863 publications that included a clinical trial registration ID. The clinical trial registry search identified 3710 trials evaluating the use of AI in clinical settings, of which 1106 trials are completed, yet only 101 trials have published results. By linking the trials to their corresponding publications, we found that 26 trials had results available from both registries and publications. There were more results in trial registries than in articles, but researchers showed a clear preference for rapid dissemination of results through peer-reviewed articles (37.6% published within one year) over trial registries (15.8%). Discrepancies and omissions of results were common, and no complete agreement was observed between the two sources. Selective reporting of publications occurred in 53.6% of cases, and the underestimation of the incidence of adverse events is alarming.

Interpretation: This research uncovers concerns with the registration and reporting of AI clinical trial results. While trial registries and publications serve distinct yet complementary roles in disseminating research findings, discrepancies between them may undermine the reliability of the evidence. We emphasise adherence to guidelines that promote transparency and standardisation of reporting, especially for investigator-initiated trials (IITs).

Funding: The authors declare no source of funding.