Do you see what i see? A more realistic eyewitness sketch recognition

Abstract

Face sketches have been used in eyewitness testimonies for about a century. However, 30 years of research shows that current eyewitness testimony methods are highly unreliable. Nonetheless, current face sketch recognition algorithms assume that eyewitness sketches are reliable and highly similar to their respective target faces. As proven by psychological findings and a recent work on face sketch recognition, these assumptions are unrealistic and therefore, current algorithms cannot handle real world cases of eyewitness sketch recognition. In this paper, we address the eyewitness sketch recognition problem with a two-pronged approach. We propose a more reliable eyewitness testimony method, and an accompanying face sketch recognition method that accounts for realistic assumptions on sketch-photo similarities and individual eyewitness differences. In our eyewitness testimony method we first ask the eyewitness to directly draw a sketch of the target face, and provide some ancillary information about the target face. Then we build a drawing profile of the eyewitness by asking him/her to draw a set of face photos. This drawing profile implicitly contains the eyewitness' mental bias. In our face sketch recognition method we first correct the sketch for the eyewitness' bias using the drawing profile. Then we recognize the resulting sketch based on an optimized combination of the detected features and ancillary information. Experimental results show that our method is 12 times better than the leading competing method at Rank-1 accuracy, and 6 times better at Rank-10. Our method also maintains its superiority as gallery size increases.