An Experimental Study of Text Preprocessing Techniques for Automatic Short Answer Grading in Indonesian

The preprocessing phase in information retrieval is intended to reduce the size of the text. Previous studies have used many preprocessing techniques in several applications such as Clustering, Classification, Document Indexing, Summarization, and Automatic Essay Grading. In this study we aim to conduct an experimental study to measure the effectiveness of preprocessing techniques in Automatic Short Answer Grading (ASAG) using questions and answers in Indonesian. As previously known, Indonesian has a different morphology from English. With the limitations of Indonesian language processing tools, we are working on several processing techniques that can be used, such as Case Folding, Tokenization, Punctuation Removal, Stopword Removal, and Stemming. We use data consisting of 6 questions and each question answered by 32 students. As a reference answer, we will use one teacher’s answer on each question. Technically, we conducted two types of experimental studies. In the first experiment, we carried out two types of pre-processing techniques, namely Punctuation Removal and Tokenization. In the second experiment, we added three other preprocessing techniques, namely Case Folding, Stemming, and Stopword Removal. We measure the similarity values of teacher and student answers using the Cosine Similarity method. Next, we calculated the correlation values and Mean Absolute Error to measure the effectiveness of the preprocessing techniques that have been used. In the end, the results of the paired-samples t-test showed that there were no significant differences in the two experiments.