A lot of talk and a badge: An exploratory analysis of personal achievements in GitHub

Context:

GitHub has introduced a new gamification element through personal achievements, whereby badges are unlocked and displayed on developers’ personal profile pages in recognition of their development activities.

Objective:

In this paper, we present an exploratory analysis using mixed methods to study the diffusion of personal badges in GitHub, in addition to the effects and reactions to their introduction.

Method:

First, we conduct an observational study by mining longitudinal data from more than 6,000 developers and performed correlation and regression analysis. Then, we conduct a survey and analyze over 300 GitHub community discussions on the topic of personal badges to gauge how the community responded to the introduction of the new feature.

Results:

We find that most of the developers sampled own at least a badge, but we also observe an increasing number of users who choose to keep their profile private and opt out of displaying badges. Additionally, badges are generally poorly correlated with developers’ skills and dispositions such as timeliness and desire to collaborate. We also find that, except for the Starstruck badge (reflecting the number of followers), their introduction does not have an effect. Finally, the reaction of the community has been in general mixed, as developers find them appealing in principle but without a clear purpose and hardly reflecting their abilities in the current form.

Conclusions:

We provide recommendations to the designers of the GitHubplatform on how to improve the current implementation of personal badges as both a gamification mechanism and as sources of reliable cues for assessing the abilities of developers.