Unsupervised Domain Extension for Nighttime Semantic Segmentation in Urban Scenes

Abstract

: This paper deals with the problem of semantic image segmentation of street scenes at night, as the recent advances in semantic image segmentation are mainly related to daytime images. We propose a method to extend the learned domain of daytime images to nighttime images based on an extended version of the CycleGAN framework and its integration into a self-supervised learning framework. The aim of the method is to reduce the cost of human annotation of night images by robustly transferring images from day to night and training the segmentation network to make consistent predictions in both domains, allowing the usage of completely unlabelled images in training. Experiments show that our approach significantly improves the performance on nighttime images while keeping the performance on daytime images stable. Furthermore, our method can be applied to many other problem formulations and is not specifically designed for semantic segmentation.