Realtime Hand Gesture Recognition System for Human Computer Interaction

Abstract

Humans are using various devices for interacting with the system like mouse, keyboard, joystick etc. We have developed a real time human computer interaction system for virtual mouse based on the hand gestures. The system is designed in 3 modules as detection of hand, recognition of gestures and human computer interaction with control of mouse events to achieve the higher degree of gesture recognition. We first capture the video using the built-in webcam or USB webcam. Each frame of hand is recognized using a media Pipe palm detection model and using opencv fingertips. The user can move the mouse cursor by moving their fingertip and can perform a click by bringing two fingertips to close. So, this system captures frames using a webcam and detects the hand and fingertips and clicks or moves of the cursor. The system does not require a physical device for cursor movement. The developed system can be extended in other scenarios where human-machine interaction is required with more complex command formats rather than just mouse events.