DeepHSAR: Semi-supervised fine-grained learning for multi-label human sexual activity recognition

Abstract

The identification of sexual activities in images can be helpful in detecting the level of content severity and can assist pornography detectors in filtering specific types of content. In this paper, we propose a Deep Learning-based framework, named DeepHSAR, for semi-supervised fine-grained multi-label Human Sexual Activity Recognition (HSAR). To the best of our knowledge, this is the first work to propose an approach to HSAR. We also introduce a new multi-label dataset, named SexualActs-150k, containing 150k images manually labeled with 19 types of sexual activities. DeepHSAR has two multi-label classification streams: one for global image representation and another for fine-grained representation. To perform fine-grained image classification without ground-truth bounding box annotations, we propose a novel semi-supervised approach for multi-label fine-grained recognition, which learns through an iterative clustering and iterative CNN training process. We obtained a significant performance gain by fusing both streams (i.e., overall F1-score of 79.29%), compared to when they work separately. The experiments demonstrate that the proposed framework explicitly outperforms baseline and state-of-the-art approaches. In addition, the proposed framework also obtains state-of-the-art or competitive results in semi-supervised multi-label learning experiments on the NUS-WIDE and MS-COCO datasets with overall F1-scores of 75.98% and 85.17%, respectively. Furthermore, the proposed DeepHSAR has been assessed on the NPDI Pornography-2k video dataset, achieving a new state-of-the-art with 99.85% accuracy.