CANAIRI: the Collaboration for Translational Artificial Intelligence Trials in healthcare

Abstract

Over the past few years, authoritative, trustworthy guidance for the clinical translation of artificial intelligence (AI) has formed mainly around two areas: responsible model development and validation; and prospective clinical trials. Initial work focused on building a good model, which generally means a model that demonstrates good performance, addresses an important clinical task, trains on the right data to address that task and could be used for some meaningful goal¹. The model should then be assessed against sets of unseen cases to ensure it can generalize beyond the test set and can potentially be tested externally. Further practices are emerging around ongoing monitoring to detect model drift and performance changes. Collectively, these practices are characterized as responsible machine learning.

However, the distinctions between the in silico context and the clinical environment are substantial, which highlights the need for clinical evaluations. In 2020, the SPIRIT-AI and CONSORT-AI reporting guidelines were published to establish the minimum reporting criteria for the conduct of prospective, interventional clinical trials evaluating the impact of an AI model^2,3. The DECIDE-AI guidelines were published shortly thereafter to address first-in-human feasibility trials of AI tools⁴. Regulatory frameworks emphasize the importance of clinical evidence, but precisely what kind and degree of evidence is needed for the approval of clinical AI applications is a matter of ongoing uncertainty⁵.