Crowdsourcing with Fewer Workers

Abstract

The crowdsourcing mechanism, as an effective and economic alternative in data science study, has attracted substantial research interests in recent years. However, despite of numerous successful crowdsourcing applications, many crowdsourcers still suffer from the high cost issue. This is mainly due to the fact that most of the crowd workers are usually not professional experts, and thus, in order to ensure the quality of crowdsourcing, extra investment is introduced by hiring many crowd workers to work on every task multiple times to inhabit the noisy submissions. In this article, we propose an approach that can mitigate the crowdsourcer's investment by using less human efforts while at the same time still has provable guarantees for the refined results. Experimental results on two real-world datasets are inspiring and consistent with the theoretical analysis.