Using gamification to create and label photos that are challenging for computer vision and people

Abstract

It would be hard to overstate the importance of Computer Vision (CV), applications of which can be found from self-driving cars, through facial recognition to augmented reality and the healthcare industry. Recent years have witnessed dramatic progress in visual-object recognition, partially ascribable to the availability of labeled data. Unfortunately, recognition of obscure, unclear and ambiguous photos that are taken from unusual angles or distances remains a major challenge, as recently shown by the creation of the ObjectNet [1]. This paper complements that work via a game in which obscure, unclear and ambiguous photos are collaboratively created and labeled by the players, who adopt the role of detectives collecting evidence against in-game criminals. The game rules enforce the creation of images that are challenging to identify for CV and people alike, as a means of ensuring the high quality of players' input.