A comparative study benchmarking colon polyp with computer-aided detection (CADe) software

Background and aims

Computer-aided detection software (CADe) has shown promising results in real-time polyp detection, but a limited head-to-head comparison of the available CADe systems has been performed. Moreover, such systems have not been compared to endoscopists using standardized videos. This study aims to compare the performance of three CADe systems in detecting polyps, employing a novel standardized methodology.

Methods

Videos from 300 colonoscopies conducted at Oslo University Hospital were analyzed. Short video clips (20–45 s) presenting normal mucosa or polyps were randomly selected. These videos were then streamed through each CADe system from Medtronic, Olympus, and Augere Medical. Each system featured diverse configurations, resulting in a total of six software settings. Sensitivity and false positivity (FP) were assessed by comparing the CADe systems to both the mean of the systems and pairwise between them. Furthermore, the systems’ performance was compared to the performance of five endoscopists.

Results

CADe systems’ sensitivity ranged between 84.9% and 98.7%, with statistically significant differences observed between the systems, both in comparison to the mean and to each other. FP rates ranged between 1.2% and 5.6%, also differing statistically significantly between the systems. The CADe systems achieving the highest sensitivity also exhibited the highest FP. Statistically significant differences in the alert delay were observed between different CADe systems and endoscopists.

Conclusions

This study highlights significant differences between commercially available CADe software regarding sensitivity and FP, but a superior performance compared to endoscopists. The software with the highest sensitivity also exhibited the highest FP, highlighting the need for further refinement.