Combining deep learning for visuomotor coordination with object identification to realize a high-level interface for robot object-picking

Abstract

We present a proof of concept to show how a deep network for end-to-end visuomotor learning to grasp is coupled with an attention focus mechanism for state-of-the-art object detection with convolutional neural networks. The cognitively motivated integration of both methods in a single robotic system allows us to realize a high-level interface to use the visuomotor network in environments with several objects, which otherwise would only be usable in environments with a single object. The resulting system is deployed on a humanoid robot, and we perform several real-world grasping experiments that demonstrate the feasibility of our approach.