Semantic segmentation considering location and co-occurrence in scene

Abstract

Semantic segmentation is a process that recognizes objects and their regions in images and is a significant challenge in image recognition. Many conventional methods have been proposed, and these studies are expected to be used for many applications such as image retrieval, robot vision for autonomous mobile robots, an automatic driving system for motor vehicles. However, semantic segmentation is one of the most difficult task because of the diversity and appearance of objects in images. This problem causes incorrect recognition not related to an image, or inconsistent with the spatial structure of the real world. We focus on understanding the scene in an image. For example, objects like “car” and “buildings” are likely to exist in the scene of street. On the other hand, those are not likely to exist in the scene of prairie. Besides, we expect that location and co-occurrence of objects are efficient information to recognize images. The region of “sky” is likely to exist in the upper part of them. In addition, “car” and “road” are likely to exist in the same image. This paper presents a method of semantic segmentation considering location and co-occurrence in the natural outdoor scene. Before recognizing objects in images, we classify them in terms of scene and execute pixel-wise object recognition. Then, we consider the location and co-occurrence of objects in the scene. Experimental results show that our proposed method is effective compared to other methods not considering scene information.