Crowdsourcing worker development based on probabilistic task network

Abstract

Crowdsourcing platforms provide an attractive solution for processing numerous tasks at low cost. However, insufficient quality control remains a major concern. In the present study, we propose a grade-based training method for workers. Our training method utilizes probabilistic networks to estimate correlations between tasks based on workers' records for 18.5 million tasks and then allocates pre-learning tasks to the workers to raise the accuracy of target tasks according to the task correlations. In an experiment, the method automatically allocated 31 pre-learning task categories for 9 target task categories, and after the training of the pre-learning tasks, we confirmed that the accuracy of the target tasks was raised by 7.8 points on average. We thus confirmed that the task correlations can be estimated using a large amount of worker records, and that these are useful for the grade-based training of low-quality workers.