Focus of attention for face and hand gesture recognition using multiple cameras

Abstract

We propose a multi-camera system that can track multiple human faces and hands as well as focus on face and hand gestures for recognition. Our current system consists of four cameras. Two fixed cameras are used as a stereo system to estimate face and hand positions. The stereo camera detects faces and hands by a standard skin color method we propose. The distances of the targets are then estimated. Next to track multiple targets, we estimate the positions and sizes of targets between consecutive frames. The other two cameras perform tracking of such targets as faces and hands. If a target is not the appropriate size for recognition, the tracking cameras acquire its zoomed image. Since our system has two tracking cameras, it can track two targets at the same time. To recognize faces and hand gestures, we propose four directional features by using linear discriminant analysis. Using our system, we experimented on human position estimation, multiple face tracking, and face and hand gesture recognition. These experiments showed that our system could estimate human position with the stereo camera and track multiple targets by using target positions and sizes even if the persons overlapped with each other. In addition, our system could recognize faces and hand gestures by using the four directional features.