Forward control of robotic arm using the information from stereo-vision tracking system

Abstract

In this paper we present the feed-forward neural network controller of robotic arm, which makes use of tracking method applied to stereo-vision cameras mounted on the head of the humanoid robot Nao, in order to touch the tracked object. The Tracking-Learning-Detection (TLD) method, which we use to detect and track the object, is known for its state-of-art performance and high robustness. This method was adjusted to be usable with a stereo-vision camera system, in order to provide 3D spatial coordinates of the object. These coordinates are used as the input for the feed-forward controller, which controls the arm of a humanoid robot. The goal of the controller is to move the hand of the robot to the object by setting arm joints into position corresponding to the object location. The controller is implemented as an artificial neural network and trained using the error back-propagation algorithm. The experiment, which demonstrates the proof of the concept, is also denoted in this paper.