Methodology for testing and monitoring AI-based software for medical diagnostics

Abstract

BACKGROUND: The global amount of investment in companies developing software based on artificial intelligence (AI) technologies for medical diagnostics was $80 million in 2016, $152 million in 2017 and is expected to continue to grow. Activity of software manufacturing companies should comply with existing clinical, bioethical, legal and methodological frameworks and standards. Both at the national and international levels, there are no uniform standards and protocols for testing and monitoring AI-based software. AIM: to develop a universal methodology for testing and monitoring AI-based software for medical diagnostics, aimed at improving its quality and implementing into practical healthcare. MATERIALS AND METHODS: During the analytical phase, a literature review was conducted on the PubMed and eLIBRARY databases. The practical stage included approbation of the developed methodology within the framework of the an Experiment on the use of innovative technologies in the field of computer vision for the analysis of medical images and further application in the health care system of the city of Moscow. RESULTS: A methodology for testing and monitoring AI-based software for medical diagnostics has been developed, aimed at improving its quality and introducing it into practical healthcare. The methodology consists of 7 stages: self-testing, functional testing, calibration testing, technological monitoring, clinical monitoring, feedback and refinement. CONCLUSION: Distinctive features of the methodology are: the cyclical stages of monitoring and software development, leading to continuous improvement of its quality, the presence of detailed requirements for the results of the software work, the participation of doctors in software evaluation. The methodology will allow both software developers to achieve high results and demonstrate achievements in various areas, and users to make an informed and confident choice among software that has passed an independent and comprehensive quality check.