Self-evaluation in advanced power searching and mapping with google MOOCs

While there is a large amount of work on creating autograded massive open online courses (MOOCs), some kinds of complex, qualitative exam questions are still beyond the current state of the art. For MOOCs that need to deal with these kinds of questions, it is not possible for a small course staff to grade students' qualitative work. To test the efficacy of self-evaluation as a method for complex-question evaluation, students in two Google MOOCs have submitted projects and evaluated their own work. For both courses, teaching assistants graded a random sample of papers and compared their grades with self-evaluated student grades. We found that many of the submitted projects were of very high quality, and that a large majority of self-evaluated projects were accurately evaluated, scoring within just a few points of the gold standard grading.