Prospective multicenter study using artificial intelligence to improve dermoscopic melanoma diagnosis in patient care

Abstract

Early detection of melanoma, a potentially lethal type of skin cancer with high prevalence worldwide, improves patient prognosis. In retrospective studies, artificial intelligence (AI) has proven to be helpful for enhancing melanoma detection. However, there are few prospective studies confirming these promising results. Existing studies are limited by low sample sizes, too homogenous datasets, or lack of inclusion of rare melanoma subtypes, preventing a fair and thorough evaluation of AI and its generalizability, a crucial aspect for its application in the clinical setting. Therefore, we assessed “All Data are Ext” (ADAE), an established open-source ensemble algorithm for detecting melanomas, by comparing its diagnostic accuracy to that of dermatologists on a prospectively collected, external, heterogeneous test set comprising eight distinct hospitals, four different camera setups, rare melanoma subtypes, and special anatomical sites. We advanced the algorithm with real test-time augmentation (R-TTA, i.e., providing real photographs of lesions taken from multiple angles and averaging the predictions), and evaluated its generalization capabilities. Overall, the AI shows higher balanced accuracy than dermatologists (0.798, 95% confidence interval (CI) 0.779–0.814 vs. 0.781, 95% CI 0.760–0.802; p = 4.0e−145), obtaining a higher sensitivity (0.921, 95% CI 0.900–0.942 vs. 0.734, 95% CI 0.701–0.770; p = 3.3e−165) at the cost of a lower specificity (0.673, 95% CI 0.641–0.702 vs. 0.828, 95% CI 0.804–0.852; p = 3.3e−165). As the algorithm exhibits a significant performance advantage on our heterogeneous dataset exclusively comprising melanoma-suspicious lesions, AI may offer the potential to support dermatologists, particularly in diagnosing challenging cases. Melanoma is a type of skin cancer that can spread to other parts of the body, often resulting in death. Early detection improves survival rates. Computational tools that use artificial intelligence (AI) can be used to detect melanoma. However, few studies have checked how well the AI works on real-world data obtained from patients. We tested a previously developed AI tool on data obtained from eight different hospitals that used different types of cameras, which also included images taken of rare melanoma types and from a range of different parts of the body. The AI tool was more likely to correctly identify melanoma than dermatologists. This AI tool could be used to help dermatologists diagnose melanoma, particularly those that are difficult for dermatologists to diagnose. Heinlein, Maron, Hekler et al. evaluate an AI algorithm for detecting melanoma and compare its performance to that of dermatologist on a prospectively collected, external, heterogeneous dataset. The AI exhibits a significant performance advantage, especially in diagnosing challenging cases.