TPSFusion: A Transformer-based pyramid screening fusion network for 6D pose estimation

Abstract

RGB-D based 6D pose estimation is a key technology for autonomous driving and robotics applications. Recently, methods based on dense correspondence have achieved huge progress. However, it still suffers from heavy computational burden and insufficient combination of two modalities. In this paper, we propose a novel 6D pose estimation algorithm (TPSFusion) which is based on Transformer and multi-level pyramid fusion features. We first introduce a Multi-modal Features Fusion module, which is composed of the Multi-modal Attention Fusion block (MAF) and Multi-level Screening-feature Fusion block (MSF) to enable high-quality cross-modality information interaction. Subsequently, we introduce a new weight estimation branch to calculate the contribution of different keypoints. Finally, our method has competitive results on YCB-Video, LineMOD, and Occlusion LineMOD datasets.