Key Frame Extraction with Face Biometric Features in Multi-shot Human Re-identification System

Abstract

Although achieving a good accuracy, the multi-shot human re-identification (MS Re-ID) experiences long processing time and large memory consumption, because it uses all of the frames it receives. The usage of all of the frames for reidentification does not only weigh the system’s process, but also gives redundant information. To enable a faster and lighter execution of the MS Re-ID system, the present study proposed a key frame extraction method using face biometric feature for MS Re-ID system. Key frames were responsible to provide important information of an individual’s face for re-identification, and avoid collecting the redundant features. The proposed key frame extraction method consists of two phases: face detection with facial landmark extraction and face tilt angle calculation. Multitask Cascaded Convolutional Networks (MTCNN) was used for face detection with facial landmark extraction, while 10 was used as the optimal number of key frames and the optimal face tilt angle range was -10° until 10°. We also compared MS Re-ID system with the proposed method to the normal single-shot and multi-shot systems to examine the proposed method’s impact on the performance. We found that the proposed key frame extraction method successfully retained 100% accuracy while reducing the memory consumption by 2% and the execution time by 19%, compared to MS Re-ID system without key frame extraction.