Computer-vision-based human-computer interaction with a back projection wall using arm gestures

A computer-vision-based interaction system for back projection walls is presented. The user controls the projected graphical user interface of an application by pointing with the arm. The mouse cursor follows the motion of the arm. Further instructions corresponding e.g. to a mouse button click can be given by voice commands. The combination of the interaction system with an application program does not require any modifications of the application software, so that the system can be used to control any software running on the back-projection wall. On the image processing and recognition level, a special emphasis has been laid on coping with images disturbed by noise and varying illumination of the environment.