TEDQuiz: Automatic Quiz Generation for TED Talks Video Clips to Assess Listening Comprehension

Abstract

In the last few years, researchers in the field of e-learning and Natural Language Processing (NLP) have shown an increased interest in automatic question generation. However, little research has discussed the automatic evaluation of listening comprehension in multimedia learning. In this work, we present an automatic quiz generation for TED Talks video clips, called TED Quiz. TED Quiz generates multiple-choice questions in two question types, gist-content questions and detail questions. We use a graph-based algorithm, Lex Rank, to identify the most important part of a talk, as the main concept of a gist-content question. We also proposed an approach to distractor selection for detail question generation that generates grammatically correct but semantically wrong sentences as distractors. The experimental results demonstrated that the measured results from automatically generated questions are comparable with that from manually generated questions because their scores were significantly correlated. Moreover, most subjects agreed that the generated listening comprehension questions were of quality and usefulness.