Scaling up Online Question Answering via Similar Question Retrieval

Faced with growing class sizes and the dawn of the MOOC, educators are in need of tools to help them cope with the growing number of questions asked in large classes since manually answering all the questions in a timely manner is infeasible. In this paper, we propose to exploit historical question/answer data accumulated for the same or similar classes as a basis for automatically answering previously asked questions via the use of information retrieval techniques. We further propose to leverage resolved questions to create test collections for quantitative evaluation of a question retrieval algorithm without requiring additional human effort. Using this evaluation methodology, we study the effectiveness of state of the art retrieval techniques for this special retrieval task, and perform error analysis to inform future directions.