Study of VR Online class for education with estimating facial expressions using deep learning (System construction and first step experiment of VR online class)

Abstract

Online lessons, wherein students attend classes online in an environment with limited interactions among the instructor and students, has become prevalent. In this study, we developed a method for online learning that uses virtual reality (VR) to make online learning more immersive and interactive. We constructed virtual online lessons using VRChat. Furthermore, we considered the online determination of the student state to be important; therefore, we developed a facial expression recognition system for use during online VR lessons, using edge computing, OpenCV, and Python. We conducted an experiment by combining VR lessons in VRChat with real-time facial detection and verified the effectiveness of the VR lessons by comparing it with conventional online lessons. By comparing the results of the facial recognition and questionnaires, it was shown that students could concentrate more on the lesson content in the VR online lesson.