Chatrooms in MOOCs: all talk and no action

We study effects of introducing a real-time chatroom into a massive open online course with several thousand students, supplementing an existing forum. The chatroom was supported by teaching assistants, and generated thousands of lines of discussion by 28\% of 681 consenting chat condition participants, mostly on-topic. Despite this, chat activity remained low ($\mu=8.2$ messages per hour) and we could find no significant effect of chat use on objective or subjective dependent variables such as grades, retention, forum participation, or students' sense of community. Further investigation reveals that only 12\% of chat participants have substantive interactions, while the remainder are either passive or have trivial interactions that are unlikely to result in learning. We also find that pervasive, highly visible chat interfaces are highly effective in encouraging both active and substantive participation in chat. When compared to chat interfaces that are restricted to a single webpage, the pervasive interface exhibits \changes{2.8 times} as many users with substantive interactions.