A Vector Space Based Approach for Short Answer Grading System

Abstract

Enhancing the quality of teaching and learning in education might be through designing, implementing, and making effective use of assessment practice. In this paper we address the task of computer assisted assessment of short student answers. We describe a new statistical approach used to design Short Answer Grading System adapted to Arabic language. The approach consists of building a semantic space that gives distributional representation of words based on word co-occurrences in text corpora. Semantic similarity is computed using the summation vector model. Score similarity is enhanced by an individual normalized term frequencies weighting and then combining the index of common words between the model and the student answers using syntactic DICE's coefficient. A great advantage of this statistical approach is that it does not require the existence of any word data models. It is particularly suitable in situations where no large, publicly available, linguistic resources can be found for a desired language. Evaluated on two datasets, the proposed approach yielded 81.49% correlation and 0.97 Root Mean Squared Error with human grading scores. The proposed approach gets significantly closer to some works in the literature and outperforms others. This shows that such an approach can be as effective as approaches using sophisticated similarities calculations that make the system difficult to achieve and to use in practice.