Uncertainty-aware correspondence identification for collaborative perception

Abstract

Correspondence identification is essential for multi-robot collaborative perception, which aims to identify the same objects in order to ensure consistent references of the objects by a group of robots/agents in their own fields of view. Although recent deep learning methods have shown encouraging performance on correspondence identification, they suffer from two shortcomings, including the inability to address non-covisibility and the inability to quantify and reduce uncertainty to improve correspondence identification. To address both issues, we propose a novel uncertainty-aware deep graph matching method for correspondence identification in collaborative perception. Our new approach formulates correspondence identification as a deep graph matching problem, which identifies correspondences based on deep graph neural network-based features and explicitly quantify uncertainties in the identified correspondences under the Bayesian framework. In addition, we design a novel loss function that explicitly reduces correspondence uncertainty and perceptual non-covisibility during learning. Finally, we design a novel multi-robot sensor fusion method that integrates the multi-robot observations given the identified correspondences to perform collaborative object localization. We evaluate our approach in the robotics applications of collaborative assembly, multi-robot coordination and connected autonomous driving using high-fidelity simulations and physical robots. Experiments have shown that, our approach achieves the state-of-the-art performance of correspondence identification. Furthermore, the identified correspondences of objects can be well integrated into multi-robot collaboration for object localization.