Dynamics of Software Development Crowdsourcing

The emergence of online labor markets has concentrated a lot of attention on the prospect of using crowdsourcing for software development, with a potential to reduce costs, improve time-to-market, and access high-quality skills on demand. However, crowdsourcing of software development is still not widely adopted. A key barrier to adoption is a lack of confidence that a task will be completed on time with the required quality standards. While good managers can develop good, intuitive estimates of task completion when assigning work to their team members, they might lack similar intuition for individuals drawn from an online crowd. The phrase, "Post and Hope" is thus sometimes used when talking about the crowdsourcing of software-development tasks. The objective of this paper is to show the value of replacing the traditional, intuitive assessment of a team's capability with a quantitative assessment of the crowd, derived through analysis of historical performance on similar tasks. This analysis will serve to transform "Post and Hope" to "Post and Expect." We demonstrate this by analyzing data about tasks performed on two popular crowdsourcing platforms: Topcoder and Upwork. Analysis of historical data from these platforms indicates that the platforms indeed demonstrate some level of predictability in task completion. We have identified certain factors that consistently contribute to task completion on both the platforms. Our findings suggest that a data-driven decision processes can play an important role in successful adoption of crowdsourcing practice for software development.