Dexterous Pre-Grasp Manipulation for Human-Like Functional Categorical Grasping: Deep Reinforcement Learning and Grasp Representations

Abstract

Many objects, such as tools and household items, can be used only if grasped in a very specific way—grasped functionally. Often, a direct functional grasp is not possible, though. We propose a method for learning a dexterous pre-grasp manipulation policy to achieve human-like functional grasps using deep reinforcement learning. We introduce a dense multi-component reward function that enables learning a single policy, capable of dexterous pre-grasp manipulation of novel instances of several known object categories with an anthropomorphic hand. The policy is learned purely by means of reinforcement learning from scratch, without any expert demonstrations. It implicitly learns to reposition and reorient objects of complex shapes to achieve given functional grasps. In addition, we explore two different ways to represent a desired grasp: explicit and more abstract, constraint-based. We show that our method consistently learns to successfully manipulate and achieve desired grasps on previously unseen object instances of known categories using both grasp representations. Training is completed on a single GPU in under three hours. Note to Practitioners—This work was motivated by the increasing popularity of robots equipped with dexterous human-like hands. Operating in environments designed for humans necessitates the ability to use human tools. That requires grasping these tools in specific ways for effective use. We propose a learning-based method to train such behaviors in highly parallelized simulation. We explore two possible ways to represent a target functional grasp: an explicit and a more abstract, constraint-based, each with its own advantages and disadvantages. Our method learns to achieve human-like behaviors in under three hours on a single computer. It successfully manipulates previously unseen object instances with both target grasp representations. Such policies could be useful for robots with human-like hands in a broad range of scenarios: household, factory or search-and-rescue, whenever there is a necessity to grasp objects in a very specific way. The main limitation of this work is that the learned behaviors were not tested in the real world. Thus, closing the sim-to-real gap is a viable direction for future work.