A user interface utilizing various contents in difficult environment

Abstract

In recent years, mobile terminals come to be used to assist works. However, we can not say that they are enough suitable to assist works either that occupy hands or that are accomplished in environment with water or fire, which cause troubles with electronic devices. In this study, in order to solve this problem, we propose an assist system using projection mapping and NUI. Note that our system is not an application program, but replaces the former UI simply. Our system consists of a depth camera, an RGB camera, and a projector. It causes a click event by detecting whether a user touches the screen. By replacing the click operation with the touching on the projected screen, it can utilize various existing applications and contents.