Question classification in Persian language based on conditional random fields

Abstract

The question classification system is one of the important subsystems in the Question Answering Systems (QAS). In such systems through retrieval methods and information extraction the texts are retrieved in order to get to a correct answer. The current study is designed to present the architecture of question classification (QC) in Persian based on the Conditional Random Fields (CRF) machine learning model and evaluate effects of various features on its accuracy. In this study, sentences were classified into two levels of coarse and fine classes based on the type of the answer to each question. After extracting features and setting sliding window on the CRF model, CRF question classifier (QC) is train. Then, the QC predicts labels for every token in question. Next, a majority voting on the question classification output, is used to extract a unique label for each question. Further, the effects of different features on the ultimate accuracy of the system were evaluated. Finally results of this question classifier, illustrate a satisfactory accuracy.