Modeling Uncertainty and Inaccuracy on Data from Crowdsourcing Platforms: MONITOR

Abstract

Crowdsourcing is characterized by the externalization of tasks to a crowd of workers. In some platforms the tasks are easy, open access and remunerated by micropayment. The crowd is very diversified due to the simplicity of the tasks, but the payment can attract malicious workers. It is essential to identify these malicious workers in order not to consider their answers. In addition, not all workers have the same qualification for a task, so it might be interesting to give more weight to those with more qualifications. In this paper we propose a new method for characterizing the profile of contributors and aggregating answers using the theory of belief functions to estimate uncertain and imprecise answers. In order to evaluate the contributor profile we consider both his qualification for the task and his behaviour during its achievement thanks to his reflection.