Development of a Smart Learning Application in Multi-person Virtual Reality Using Biometric Measures of Neuroimaging, Eye Tracking, and Haptic Interactions

Abstract

Non-text-based smart learning refers to technology-supported learning that uses non-text features (i.e. visualized information) and adapted learning materials based on the individual’s needs. Fully immersive multi-person virtual reality (MVR) refers to humans located at different places wearing VR devices to join a single virtual room, a classroom with unlimited size, to learn from an instructor. The learning environment is poised to undergo a major reformation, and MVR will augment, and possibly replace, the traditional classroom learning environment. The ultimate purpose of this research and development of apps is to discover new smart learning methodologies within the MVR environment using nonintrusive multimodal analysis of physiological measures, including eye movement characteristics, haptic interactions, and brain activities. We provide the design concepts that were developed and implemented to create the MVR semantic network app. Furthermore, demonstration details are provided showcasing how we could leverage the MVR technology for education and training.