Automated Person Identification Framework Based on Fingernails and Dorsal Knuckle Patterns

Abstract

Handimages are of paramount importance within critical domains like security and criminal investigation. They can sometimes be the only available evidence of an offender's identity at a crime scene. Approaches to person identification that consider the human hand as a complex object composed of many components are rare. The approach proposed in this paper fills this gap, making use of knuckle creases and fingernail information. It introduces a framework for automatic person identification that includes localisation of the regions of interest within hand images, recognition of the detected components, segmentation of the region of interest using bounding boxes, and similarity matching between a query image and a library of available images. The following hand components are considered: i) the metacarpohalangeal, commonly known as base knuckle; ii) the proximal interphalangeal joint commonly known as major knuckle; iii) distal interphalangeal joint, commonly known as minor knuckle; iv) the interphalangeal joint, commonly known as thumb's knuckle, and v) the fingernails. A key element of the proposed framework is the similarity matching and an important role for it is played by the feature extraction. In this paper, we exploit end-to-end deep convolutional neural networks to extract discriminative high-level abstract features. We further use Bray-Curtis (BC) similarity for the matching process. We validated the proposed approach on well-known benchmarks, the ‘11k Hands' dataset and the Hong Kong Polytechnic University Contactless Hand Dorsal Images known as ‘PolyU HD’. We found that the results indicate that the knuckle patterns and fingernails play a significant role in the person identification. The results from the 11K dataset indicate that the results for the left hand are better than the results for the right hand. In both datasets, the fingernails produced consistently higher identification results than other hand components, with a rank-1 score of 93.65% on the ring finger of the left hand for the ‘11k Hands' dataset and rank-l score of 93.81% for the thumb from the ‘PolyU HD’ dataset.