Sea View Extension for Semantic Segmentation in Cityscapes

Abstract

Semantic segmentation in computer vision is a challenging area of research, aiming to accurately segment and categorize objects and regions within an image. One widely used dataset for this task is Cityscapes, which contains a variety of city-related object classes such as cars, pedestrians, bicycles, and buildings. However, the Cityscapes dataset does not include any aquatic view classes, which limits its potential for applications in coastal and marine environments. This paper presents a novel approach to extend the Cityscapes dataset with aquatic classes to address this limitation. Our proposed method involves the implementation of two state-of-the-art neural network models, one based on the Cityscapes dataset and the other on a common aquatic dataset. We then selectively extract the aquatic segmen-tation results from the corresponding model according to the aquatic label. We further generate a mask image for the sea class and merge it precisely with the resulting image from the Cityscapes-based model. Our method is evaluated by comparing the performance of the original Cityscapes-based model with the extended Cityscapes-based model on a set of test images that contain aquatic views. The results show that our approach can maintain the original model’s high segmentation accuracy for all views except for aquatic areas while preserving the relevant parts of the marine model in terms of accuracy and area coverage. Additionally, our approach does not require retraining, thus saving computational resources and time.