An Exploratory Study of Reactions to Bot Comments on GitHub

Abstract

The widespread use of bots to support software development makes social coding platforms such as GitHub a particularly rich source of data for the study of human-bot interaction. Software development bots are used to automate repetitive tasks, interacting with their human counterparts via comments posted on the various discussion interfaces available on such platforms. One type of interaction supported by GitHub involves reacting to comments using predefined emoji. To investigate how users react to bot comments, we conducted an observational study comprising 54 million GitHub comments, with a particular focus on comments that elicited the laugh reaction. The results from our analysis suggest that some reaction types are not equally distributed across human and bot comments and that a bot's design and purpose influence the types of reactions it receives. Furthermore, while the laugh reaction is not exclusively used to express laughter, it can be used to convey humor when a bot behaves unexpectedly. These insights could inform the way bots are designed and help developers equip them with the ability to recognize and recover from unanticipated situations. In turn, bots could better support the communication, collaboration, and productivity of teams using social coding platforms.