Digit-writing hand gesture recognition by hand-held camera motion analysis

Abstract

A camera motion detection and analysis algorithm applicable to hand-held devices, such as mobile phones, has been developed and applied to digit-writing gesture recognition. The writing stroke is recorded from an image sequence taken by a moving camera. The automatic speed adaptation capability developed in the motion detection system has enabled very robust writing stroke detection. As a result, the temporal stroke distortion due to irregular writing speed has been eliminated. Since both the direction and magnitude of motion is detected at each instant, the writing stroke is correctly reconstructed by integrating the results. For this reason, feature vector for each digit character was constructed by connecting feature distribution in each direction. As a result, handwriting gesture recognition is achieved by simple template matching. The system performance has been evaluated by digit-writing gesture recognition with irregular writing speed, different users, or cursive writing. Preliminary experiments on hand-writing Chinese character recognition have also been attempted and the potentiality of the algorithm for more complicated gesture patterns has been tested.