Multi-modal data fusion of Voice and EMG data for robotic control

Abstract

Wearable electronic equipment is constantly evolving and is increasing the integration of humans with technology. Available in various forms, these flexible and bendable devices sense and can measure the physiological and muscular changes in the human body and may use those signals to machine control. The MYO gesture band, one such device, captures Electromyography data (EMG) using myoelectric signals and translates them to be used as input signals through some predefined gestures. Use of this device in a multi-modal environment will not only increase the possible types of work that can be accomplished with the help of such device, but it will also help in improving the accuracy of the tasks performed. This paper addresses the fusion of input modalities such as speech and myoelectric signals captured through a microphone and MYO band, respectively, to control a robotic arm. Experimental results obtained as well as their accuracies for performance analysis are also presented.