Gesture Recognition Model with Multi-Tracking Capture System for Human-Robot Interaction

Abstract

This study develops a wireless gesture recognition system for human-robot interaction using a high-speed marker-based motion capture system that requires no hardware development and no onboard power source. A novel gesture recognition model recognizes four gestures performed while holding a rigid-body object and translates them into robot control signals. The gestures are Flick Back to Front, Flick Front to Back, Rotate Clockwise, and Rotate Counter-clockwise. The system has four main components: a host PC with Vicon Tracker software, a set of Vicon Vantage V8 infrared cameras, a client PC that receives motion capture data and translates gestures to instruct a simulated KUKA youBot robot in the Gazebo simulation environment, and a gesture input unit. The poses of the gesture input unit are used to train the model using a surrogate deep neural network and the XGBoost ensemble method in a semi-supervised setting. The algorithm’s decision-making process is explicated through the implementation of the Layer-wise Relevance Propagation methodology in PyTorch. The control approach is similar to the way trainers teach domestic pets to perform specific actions in response to different gestures. The proposed method offers an alternative to commanding the robot through typing or using joysticks. The current gesture recognition rate is around 60%, but performance will improve over time as new training samples are collected and event detection algorithms are improved to avoid misinterpreting unrelated movements as classified gestures.