Real-time Hand Gesture Recognition for Robotic Arm and Home Automation

Abstract

Hand gestures are a symbolic and non-vocal language and are used by an individual to communicate. With computer vision, hand gestures can be detected and be used to talk with a capable computer, leading to the field of Human-Computer interconnection. The field of computer vision has been achieving cutting edge results with the advent of deep learning models. The work implements the Inception v3 architecture [1], which is a convolutional neural network. The model is retrained on our data set using Transfer learning, with which we reduce the requirements on computational resources, data and time. In this project, a hand gesture is performed in front of a web camera of a system. The gestures are predicted as one among six gestures with a corresponding probability. This project deals with the applications of the detected hand gestures in home automation and control of a robotic arm. Hand gestures are simple to perform, and it makes managing home effortless compared to manually intervening and providing instructions to a machine. In the home automation model, the gesture classification results from the system are transmitted to the microcontroller which switches on or off a home device. The robotic arm is a mechanical system which is used in manipulating the movement of lifting, moving, and placing the workpiece to lighten the work of man. It is equipped with servo motors and is controlled by our hand gestures to perform lifting and dropping of objects and rotation of the robotic arm.