Scene identification using visual semantic segmentation and supplementary classifier for resource-constrained edge systems

Abstract

This paper presents a scene identification method employing semantic segmentation where the method provides real-time computation in resource-constrained edge devices. Scene identification could be crucial for intelligent systems (e.g., service robots, drone-based inspection, and visual surveillance) regarding a proper decision making of those systems. Existing methods focus on adopting a deep learning-based image classification for the identification. However, those approaches may provide wrong identification due to an overlap of spatial features when training dataset is limited.In this paper, we propose an accurate scene identification with a novel approach. Our method includes two-steps: 1) measurement of object class frequency with visual semantic segmentation; 2) scene classification using class frequencies. For fast computation, we build a lightweight backbone network for the segmentation model in addition to TensorRT-based optimization. From the experiments, we validate that our method improves the identification accuracy by 12% compared to conventional visual classification-based method. In terms of computation, we observe that the method enables real-time inference on resource- constrained devices (i.e., NVIDIA Jetsons).