3D-IMMC: Incomplete Multi-Modal 3D Shape Clustering via Cross Mapping and Dual Adaptive Fusion

Abstract

In recent years, with the rapid growth number of multi-modal 3D shapes, it has become increasingly important to efficiently recognize a vast number of unlabeled multi-modal 3D shapes through clustering. However, the multi-modal 3D shape instances are usually incomplete in practical applications, which poses a considerable challenge for multi-modal 3D shape clustering. To this end, this paper proposes an incomplete multi-modal 3D shape clustering method with cross mapping and dual adaptive fusion, termed as 3D-IMMC, to alleviate the negative impact of the missing modal instances in multi-modal 3D shapes, thus obtaining competitive clustering results. To the best of our knowledge, this paper is the first attempt to the incomplete multi-modal 3D shape clustering task. By exploring the spatial relationship between different 3D shape modalities, a spatial-aware representation cross-mapping module is proposed to generate representations of missing modal instances. Then, a dual adaptive representation fusion module is designed to obtain comprehensive 3D shape representations for clustering. Extensive experiments on the 3D shape benchmark datasets (i.e., ModelNet10 and ModelNet40) have demonstrated that the proposed 3D-IMMC achieves promising 3D shape clustering performance.