Image4Assess: Automatic learning processes recognition using image processing

Abstract

Recently, there has been a growing interest in improving students' competitiveness in STEM education. Self-reporting and observation are the most used tools for the assessment of STEM education. Despite their effectiveness, such assessment tools face several challenges, such as being labor-intensive and time-consuming, prone to subjective awareness, depending on memory limitations, and being influenced due to social expectations. To address these challenges, in this research, we propose an approach called Image4Assess that---by benefiting from state-of-the-art machine learning like convolutional neural networks and transfer learning---automatically and uninterruptedly assesses students' learning processes during STEM activities using image processing. Our findings reveal that the Image4Assess approach can achieve accuracy, precision, and recall higher than 85% in the learning process recognition of students. This implies that it is feasible to accurately measure the learning process of students in STEM education using their imagery data. We also found that there is a significant correlation between the learning processes automatically identified by our proposed approach and students' post-test, confirming the effectiveness of the proposed approach in real-world classrooms.