Addressing challenges in industrial pick and place: A deep learning-based 6 Degrees-of-Freedom pose estimation solution

Object picking is a fundamental, long-lasting, and yet unsolved problem in industrial applications. To complete it, 6 Degrees-of-Freedom pose estimation can be crucial. This task, easy for humans, is a challenge for machines as it involves multiple intelligent processes (for example object detection, recognition, pose prediction). Pose estimation has recently made huge steps forward, due to the advent of Deep Learning. However, in real-world applications it is not trivial to compute it: each use-case needs an annotated dataset and a model robust enough to face its specific challenges. In this paper, we present a comprehensive investigation focused on a specific use-case: the picking of four industrial objects by a collaborative robot’s arm, addressing challenges related to reflective textures and pose ambiguities of heterogeneous shapes. Thus, Artificial Intelligence is crucial in this process, utilizing Convolutional Neural Networks to discern an object’s pose by extracting hierarchical features from a single image. In detail, we propose a new synthetic dataset of industrial objects and a fine-tuning method to close the sim-to-real domain gap. In addition, we improved an existing pipeline for pose estimation and introduced a new version of an existing method, based on Convolutional Neural Networks. Finally, extensive experiments were conducted with a Universal Robot UR5e. Results show our strategy achieves good performances with an average successful picking rate of 75% on these new objects. Considering the lack of available datasets for pose estimation, coupled with the significant time and labor required for annotating new images, we contribute to the scientific community by providing a comprehensive dataset, and the associated generation and estimation pipelines.¹