Correlating skill and improvement in 2 MOOCs with a student's time on tasks

Because MOOCs offer complete logs of student activities for each student there is hope that it may be possible to find out which activities are the most useful for learning. We start this quest by examining correlations between time spent on specific course resources and various measures of student performance: score on assessments, skill as defined by Item Response Theory, improvement in skill over the period of the course, and conceptual improvement as measured by a pre-post test. We study two MOOCs offered on edX.org by MIT faculty: Circuits and Electronics (6.002x) and Mechanics Review (8.MReV). Surprisingly, we find strong negative correlations in 6.002x between student skill and resource use; we attribute these findings to the fact that students with higher initial skills can do the exercises faster and with less time spent on instructional resources. We find weak or slightly negative correlations between relative improvement and resource use in 6.002x. The correlations with learning are stronger for conceptual knowledge in 8.MReV than with relative improvement, but similar for all course activities (except that eText checkpoint questions correlate more strongly with relative improvement). Clearly, the wide distribution of demographics and initial skill in MOOCs challenges us to isolate the habits of learning and resource use that correlate with learning for different students.