Learning object-specific vision-based manipulation in virtual environments

Abstract

In this paper a method for learning object-specific vision-based manipulation is described. The proposed approach uses a virtual environment containing models of the objects and the manipulator with an eye-in-hand camera to simplify and automate the training procedure. An object with a form that requires a unique final gripper position and orientation was used to train and test the implemented algorithms. A series of smooth paths leading to the final position are generated based on a typical path defined by an operator. Images and corresponding manipulator positions along the produced paths are gathered in the virtual environment and used for the training of a vision-based controller. The controller uses a structure of radial-basis function (RBF) networks and has to execute a long reaching movement that guides the manipulator to the final position so that afterwards only minor justification of the gripper is needed to complete the grasp.