Real-time human detection and tracking from a mobile armed robot using RGB-D sensor

Abstract

This paper presents a prototype of a military robot which implements an application of real time detection and tracking of human beings. A depth sensing camera system by Microsoft Kinect is utilized on a physical robot for human detection and aiming in both static and dynamic modes of operation. Human detection in the static mode is obtained by using APIs of Open Natural Interaction (OpenNI) framework and the aiming routine is carried out using Regression Analysis between image pixels and the gun control motors. In the dynamic mode, human detection is done by implementing several algorithms such as voxelization of point clouds, RANSAC, and Histogram of Oriented Gradients in Point Cloud Library and the aiming routine is carried out using inverse kinematics.