Early facial expression recognition using early RankBoost

Abstract

This work investigated a new challenging problem: how to recognize facial expressions as early as possible, in contrast to finding ways to improve the facial expression recognition rate. Unlike conventional facial expression recognition, early facial expression recognition is inherently difficult due to the initial low intensity of the expressions. To overcome this problem, a novel early recognition approach based on RankBoost is used to infer the facial expression category of an input facial expression sequence as early as possible. Facial expression intensity increases monotonically from neutral to apex in most cases, and this observation was elaborated for developing an early facial expression recognition method. To identify the most discriminative features of subtle facial expressions, weak rankers are used to learn the temporal variations of pairwise subtle facial expression features in accordance with their temporal order. Then, a weight propagation method is applied to boost a weak ranker into an early recognizer. Experiments on the Cohn-Kanade database and a custom-made dataset built using a high-speed motion capture system demonstrated that the proposed method has promising performance for early facial expression recognition.