Automatic Nucleus Detection of Pap Smear Images using Stacked Sparse Autoencoder (SSAE)

Abstract

Pap smear image analysis is an effective and common way for early diagnosis of cervical cancer. Nucleus and cytoplasm morphology analysis are main criterion in determining whether the cells are normal or abnormal. Therefore, the accuracy of nucleus detection is crucial before further analysis of cell changes. One of the main problem in automatic nucleus detection process on pap smear image is how to accurately detect the nucleus on multi-cell image which usually contain overlapped cells. To solve the problem, authors propose a deep learning (DL) approach in particular Stacked Sparse Autoencoder (SSAE) as a feature representation process in multi-cell pap smear images. SSAE is able to capture high level feature through learning processing from low level feature (pixel). The high level feature will be a differentiator feature between nucleus and non-nucleus. In this research, authors have applied sliding window operation (SWO) on pap smear images and utilized softmax classifier (SMC) for the nucleus classification process. The main purpose in this research is to measure the performance of SSAE+SMC for the detection of nucleus on overlapped cells. The result shows that fine-tuned SSAE+SMC has significantly increased the accuracy of nucleus detection. The best accuracy achieves 0.876 on 50 x 50 window size.