Dynamic spatial augmented reality with a single IR camera

Abstract

We propose a dynamic spatial augmented reality (SAR) system with effective machine learning techniques and edge-based object tracking. Real-time 3D pose estimation is the significant problem of projecting images on moving objects. However, camera-based feature detection is difficult, because most targets have a texture-less surface. Image projection and projected images also interfere with detection. Obtaining 3D shape information with stereo-paired cameras [Resch et al. 2016] is still a time-consuming process, and using a depth sensor with IR [Koizumi et al. 2015] is still unstable and have a fatal time-delay for the dynamic SAR. Therefore, we quickly and robustly estimate the 3D pose of the target objects by using effective machine learning with IR images. And by the combined use of high-speed edge-based object tracking, we realize a stable and low-delay SAR for moving objects.