Abstract LB015: Clinical evaluation of The Paris System-based artificial intelligence algorithm for reporting urinary cytopathology

BACKGROUND: The Paris System (TPS) for Reporting Urinary Cytology provides standardized diagnostic criteria for urinary tract cytology specimens, focusing on the detection of high-grade urothelial carcinoma (HGUC). Since the publication in 2016, numerous studies have reported a decrease in atypical diagnosis and a significant improvement in the detection of HGUC after adopting TPS. However, the major challenges include labor-intensive screening and interobserver variations. Artificial intelligence (AI) in medical imaging analysis is an emerging tool for ancillary diagnosis. To this end, we have developed an AI algorithm and conducted a retrospective study to evaluate the AI-assisted urine cytology reporting workflow. METHODS: A total of 131 urine cytology slides from bladder cancer patients, either first diagnosis or post-treatment follow-up, were retrieved and digitized as whole slide images (WSIs). A deep learning-based computational model was used to analyze these WSIs. Candidate urothelial cells were automatically highlighted and classified into high-risk and low-risk atypia categories in each sample based on TPS criteria. Slide-wide statistical data, including a total number of high-risk and low-risk cells, nuclear-cytoplasmic ratio (N:C ratio) and nuclear area for each cell, and the distribution and mean values of these variables, were also provided. In a blind study, a cytotechnologist and a cytopathologist parallelly reviewed the AI-annotated images and quantitative data for each WSI sample. Suspicious for HUGC and HGUC were considered to be "positive" and the other diagnostic categories were considered to be "negative" according to whether trigger cystoscopy. The results were compared with the final diagnosis reviewed by a senior cytopathologist via microscopy to evaluate the performance of the AI-assisted model. RESULTS: There were 35 positive and 96 negative urine cytology samples based on the final diagnosis. The AI algorithm annotated a total of 26,502 cells and a mean of 757.2 cells at cancer risk from all positive samples and a total of 950 cells and a mean of 9.9 cells at cancer risk from all negative samples. The mean N:C ratio was 0.68 for high-risk atypical cells and 0.56 for low-risk atypical cells. The performance of the AI-assisted reports of the cytotechnologist was 88.6% sensitivity, 97.9% specificity, 93.9% positive prediction value (PPV), and 95.9% negative prediction value (NPV) and the cytopathologist was 91.4% sensitivity, 95.8% specificity, 88.9% PPV, and 96.8% NPV. CONCLUSIONS: We demonstrated an AI algorithm that can effectively assist the reporting of urine cytology by classifying urothelial cells at cancer risk and calculating quantitative data using WSI analysis. Integrating this AI model into clinical urine cytology workflow supported TPS for reporting urinary cytopathology, reduced the interobserver variations, and may potentially reduce the human labor for screening. Citation Format: Wei-Lei Yang, Jen-Fan Hang, Chi-Bin Li, Ching-Ming Lee, Yi-Sheng Lin, Tang-Yi Tsao, Ming-Chen Chang, Yen-Chuan Ou, Tien-Jen Liu. Clinical evaluation of The Paris System-based artificial intelligence algorithm for reporting urinary cytopathology [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr LB015.