A real time micro-expression detection system with LBP-TOP on a many-core processor

Abstract

The implementation of a micro-expression detection system introduces challenges to sustain a real time recognition result. In order to surmount these problems, this paper examines the algorithm of a serial Local Binary Pattern from Three Orthogonal Planes (LBP-TOP) in order to identify the performance limitations for real time system. Videos from SMIC and CASMEII were up sampled to higher resolutions (280×340, 560×680 and 1120×1360) to cater the need of real life implementation. Then, a parallel multicore-based LBP-TOP algorithm is studied as a benchmark. Experimental results show that the parallel LBP-TOP algorithm exhibits 7× and 8× speedup against serial LBP-TOP for SMIC and CASMEII database respectively for the highest tested video resolution utilising 24- logical processor multi-core architecture. To further reduce the computational time, this paper also proposes a many-core parallel LBP-TOP algorithm using Compute Unified Device Architecture (CUDA). In addition, a method is designed to calculate the threads and blocks required to launch the kernel when processing videos from different resolutions. The proposed algorithm increases the performance speedup to 117× and 130× against the serial algorithm for the highest tested resolution videos.