Crowdsourcing with Diverse Groups of Users

Abstract

When crowdsourcing to achieve some goal, or to gather information, there is a distinct advantage to choosing a diverse team of users. Past research has shown the advantages of diversity in the workplace, as team members bring different perspectives and points of view. Similarly, when choosing users from a crowd, user diversity must be taken into consideration. This paper studies the diverse team formation problem. More precisely, we are given a set of required skills, as wells as a large set of people, each of who has some subset of the skills. The goal is to form a team satisfying the skills, that is also diverse, as is reflected by differences in the characteristics of team members (e.g., gender, race, country of residence, economic bracket). We show that finding an optimal (diverse) team of people is an NP-complete problem. In practice, the number of candidates is likely to strongly dominate the number of skills and characteristics. Hence, we provide an algorithm that returns an optimal solution, while running in time that is indifferent to the number of candidates (but is exponential in the number of skills and characteristics). We also provide a polynomial method for approximating optimal team formation by a reduction to the problem of submodular function maximization with a matroid constraint. Extensive experimentation shows both scalability of our methods, and the quality of the solutions returned.