Event Camera Survey and Extension Application to Semantic Segmentation

Abstract

Event cameras are a kind of radically novel vision sensors. Unlike traditional standard cameras which acquire full images at a fixed rate, event cameras capture brightness changes for each pixel asynchronously. As a result, the output of event camera is a stream of events, which include information of each pixel about the time, location and sign of brightness changes. Event cameras have many advantages over traditional cameras: high temporal resolution (with microsecond resolution), low latency, low power (10mW), high dynamic range (HDR>120 dB). Therefore, event cameras are increasingly used in the field in which many problems cannot be solved due to limitation of frame-based cameras, such as AR/VR, video game, mobile robotics and computer vision. In this paper, we first describe the basic principle and advantageous properties of event camera. Additionally, we introduce wide range of applications of event camera. Specific functions: tracking, high speed and high dynamic range video reconstruction, dynamic obstacle detection and avoidance, motion segmentation. Based on these fundamental applications, much more intelligent and even completely vision-based application are produced, like its combination with fully convolutional network. Finally, we use DeepLab to do semantic segmentation of the scene and apply this result to the corresponding points of the 3D reconstruction of the same scene. We also propose potential solution to solve the ambiguity problem of semantic segmentation in the end.