A User-Centered Approach to Gamify the Manual Creation of Training Data for Machine Learning

Abstract The development of artificial intelligence, e. g. for Computer Vision, through supervised learning requires the input of large amounts of annotated or labeled data objects as training data. Usually, the creation of high-quality training data is done manually which can be repetitive and tiring. Gamification, the use of game elements in a non-game context, is one method to make such tedious tasks more interesting. We propose a multi-step process for gamifying the manual creation of training data for machine learning purposes. In this article, we give an overview of related concepts and existing implementations and present a user-centered approach for a real-life use case. Based on a survey within the target user group we identified annotation use cases and dominant player characteristics. The results served as a foundation for designing the gamification concepts which were then discussed with the participants. The final concept includes levels of increasing difficulty, tutorials, progress indicators and a narrative built around a robot character which at the same time is a user assistant. The implemented prototype is an extension of an existing annotation tool at an AI product company and serves as a basis for further observations.