Sensor Level Fusion for Multi-modal Biometric Identification using Deep Learning

In this paper, a new multi-modal biometric identification system is proposed using a Convolutional neural network (CNN), in which we make an early fusion (sensor level fusion) of face, palmprint, and iris by stacking the three biometric like RGB channels of an image, then used as input to CNN. This approach uses four popular pretrained deep-convolutional neural network (CNN) models, which are Inceptionv3, GoogleNet, ResNet18, and SqueezeNet, to make a robust and fast classification. Also, it avoids training a new model from scratch that needs lots of data and calculations. So, we explore the pretrained deep-convolutional neural network by two strategies: feature extraction and fine-tuning. In the first strategy, the pre-trained deep-convolutional neural network (CNN) models are used as feature extractors, and in the second one, the pretrained SqueezeNet model is adopted to our task with 152 classes instead of the ImagenNet classification with 1000 classes. The experimental results of the proposed multi-modal biometric system achieve promising accuracy.