SSFD-Net: Shared-Specific Feature Disentanglement Network for Multimodal Biometric Recognition with Missing Modality

Abstract

Currently, multimodal biometric recognition is one of the safest identity recognition technologies. Most existing multimodal biometric recognition algorithms require test samples with complete multimodal data. However, in practical scenarios, it is difficult to obtain complete multimodal biometric data due to factors such as collection equipment and environment. To address this problem, we proposed a Shared-Specific Feature Disentanglement Network (SSFD-Net) for palmprint and palmvein based multimodal biometric recognition with missing modality. Firstly, the multimodal shared-specific feature disentanglement network with inter-modality triplet loss and inter-modality identity consistency loss is proposed to split each modality into shared and specific features. Secondly, the cross-modal feature transformation module is designed to establish the correlation of specific features for different modalities. Subsequently, the features of missing modality are reconstructed by fusing the shared features from available modalities and specific features generated by the cross-modal feature transformation module. Besides, the intra-modality identity consistency loss is presented to enhance the discriminative ability of reconstructed features of missing modality. Experimental results demonstrate that our proposed model outperforms state-of-the-art incomplete multimodal learning models on three multimodal biometric benchmark datasets.