An Interaction System Using Speech and Gesture Based on CNN

Abstract

Nowadays, Hand gestures playing a important role for human interactions with the computer. Deep Learning is a part of machine learning methods which makes the recognition process easier by using Convolution Neural Networks (ConvNet/CNN). Convolution Neural Networks is a multilayer process network which includes Input layer, Convolution layer, Max pooling layer, Fully connected layer, Output layer. When compared to other algorithms, CNN can give more accurate results. CNN is mainly used to analyze visual images and for the image processing, segmentation and classification with higher accuracy. Here, this model consists of two main systems. One is voice input is converted into text and hand gestures and second approach is hand gestures conversion to text. These two systems are mainly used for abnormal people. These systems are implemented in Python and OpenCV is used to capture images. Each of these two systems has different modules. Human Computer Interaction are main source for the communication between humans and computer. So, these systems are helpful in communicating some information to humans. These systems are free from lighting conditions and background noise by using CNN algorithm.