A system for controlling personal computers by hand gestures using a wireless sensor device

Abstract

There are a lot of home appliances and personal computers around us. However, few of the user interfaces are designed on user-centric approaches. In this study, as an interface focusing on the ease of use, we develop a system to control personal computer by applying the natural behavior of human. Research issues include the definition of the association between the PC operations and gestures, the recognition of hand gestures, the adjustment of the error of the gestures, and how to realize the system. We define the association of the PC operations used very often with hand gesture of human. Dynamic Time Warping (DTW) algorithm is used to recognize gestures. Magic Ring (MR) is a finger-worn ring type sensor used to collect data of gestures. The MR collects the acceleration value of a gesture by acceleration sensor installed in the MR and transmits the value to a PC through wireless sensor installed in the MR. On the PC side, control commands of gestures are managed. In order to perform the correct PC operations by the gestures avoiding malfunction, a bending sensor which is used for detecting the start of gesture is employed. In this system, more than 20 kinds of hand gestures can be recognized by DTW-based method. We have experimented with controlling typical applications installed in a PC, by hand gestures for evaluating our system.