Understanding Confounding Factors in Face Detection and Recognition

Currently, face recognition systems perform at or above human-levels on media captured under controlled conditions. However, confounding factors such as pose, illumination, and expression (PIE), as well as facial hair, gender, skin tone, age, and resolution, can degrade performance, especially when large variations are present. We utilize the IJB-C dataset to investigate the impact of confounding factors on both face detection accuracy and face verification genuine matcher scores. Since IJB-C was collected without the use of a face detector, it can be used to evaluate face detection performance, and it contains large variations in pose, illumination, expression, and other factors. We also use a linear regression model analysis to identify which confounding factors are most influential for face verification performance.