Reproducibility of cytological diagnoses in evaluating liquid cervical smears and immunocytochemical co-expression of p16/Ki-67 using manual and automatic methods

Aim. To assess the reproducibility of cytological diagnoses in evaluating liquid cervical smears and immunocytochemical co-expression of p16/Ki-67 using manual and automatic methods. Materials and methods. Cytological smears prepared using the liquid cytology method on the Becton Dickinson device (SurePath technology) were studied. An immunocytochemical study was carried out using a Ventana BenchMark Ultra automatic immunostainer with a commercial CINtec kit (determination of p16/Ki-67 co-expression). In total, 100 cytological slides (50 pairs of Pap-smears and immunocytochemical slides) were studied. The diagnostic kit was reviewed by five cytologists independently, and the cytologic slides were evaluated using four categories according to the Bethesda system (2014) and according to the categories of normal/abnormal. The co-expression of p16/Ki-67 was assessed per the manufacturer's recommendations (Roche) using the manual method (light microscope) and the automatic Vision Cyto Pap ICC system. Statistical processing of the results was performed using the SPSS software package version 26.0.0.0 with the calculation of the reproducibility indices of Cohen's kappa and Fleiss' kappa. Results. When assessing the reproducibility of four categories of cytological diagnoses according to the Bethesda system (2014), Cohen's kappa was 0.0480.265. The overall Fleiss' kappa between all cytologists was 0.103. When only two categories (normal/abnormal) were used, the reproducibility ranged from 0.058 to 0.377. When assessing the co-expression of p16 and Ki-67, Cohen's kappa reproducibility was from 0.196 to 0.574, while the overall Fleiss' kappa was 0.407. When comparing the evaluation results of each of the cytologists with the neural network, Cohen's kappa reproducibility ranged from 0.103 to 0.436. Conclusion. The reproducibility of cytological diagnoses according to the Bethesda system (2014) and two categories (normal/abnormal) based on the Pap smear study was low. Such results are primarily due to a large number of abnormal smears in the study. The immunocytochemical method has diagnosis reproducibility three times higher, indicating the need to measure the co-expression of p16 and Ki-67 to increase the sensitivity and specificity of the cytological method. Similar reproducibility when comparing the manual and automatic evaluation of the "double label" suggests that the neural network algorithm can currently help in decision support rather than replace the cytologist at the diagnostic stage.