Comparing the performance of machine learning and deep learning algorithms classifying messages in Facebook learning group

The use of computer-mediated communication (CMC) has been ubiquitous in higher education. To better understand students’ behaviors and facilitate students’ learning through CMC, this study aimed to classify messages in Facebook learning group which was created as an on-line discussion board. Different machine learning and deep learning classification models were proposed, trained and testified with corpuses from PTT, one of the famous on-line forums in Taiwan. Furthermore, the classification of Facebook messages by these well-trained models were compared with human coding. Results revealed that recurrent neural network (RNN) with word to vector (W2V) for feature extraction demonstrated the best performance in accuracy. In addition, the combination of RNN and TF-IDF was proved to have the highest correlation with human work. Implications for artificial intelligence (AI) in education context was discussed.