Evaluation of an AI system designed for breast cancer detection

Screening programs for the early detection of breast cancer have significantly reduced mortality in women. The limitations of these programmes are primarily due to the use of 2D techniques and the high number of mammograms to be read by radiologists. Artificial Intelligence (AI) systems may lead to new tools to help radiologists read mammograms and classify the examination based on the malignancy of the detected lesions. Several factors related to breast characteristics (thickness and density), technical factors of image acquisition, X-ray system performance and image processing algorithms can influence the outcome of a mammogram and thus also the detection capability of an AI system. The aim of this work is to analyze the robustness of an AI system for breast cancer detection and its dependence on breast characteristics and technical factors. For this purpose, mammograms from a population-based screening program were scored with the AI system. The AUC (area under the ROC curve) index generated from the scoring ROC curve was 0.92 (CI(95%) = 0.89 - 0.95), demonstrating the robust performance of the AI system. Moreover, the statistical analysis performed showed that the AUC index was independent of breast characteristics, the type of mammographic system and most of the technical parameters considered, demonstrating the effectiveness of the AI system.