A Novel Mutual Information-Based Approach for Neurophysiological Characterization of Sense of Presence in Virtual Reality

Abstract

Objective: The presented work aimed to investigate neurophysiological markers of sense of presence in virtual reality (VR). The study was based on developing and preliminary validating a neurophysiological-based approach for sense of presence evaluation. Methods: A VR environment was designed to systematically modulate multisensory conditions, including visual, auditory, and vibrotactile stimuli. EEG, ECG, and EDA signals were recorded. The Mutual Information-based sense of presence index ($\text{SoP}_{\text{MI}}$) was developed as a synthetic metric for sense of presence, integrating multiple physiological signals. Signal preprocessing and analysis were conducted using EEG-based Global Field Power (GFP) and Skin Conductance Level (SCL), among other parameters, to explore their relationship with sense of presence under different VR conditions. Results: The $\text{SoP}_{\text{MI}}$ index demonstrated sensitivity to varying levels of multisensory integration and immersion (all p < 0.001). EEG-derived features, particularly in theta and alpha bands, were highly correlated with subjective sense of presence scores (R = 0.559, p < 0.007). Additionally, autonomic markers, such as skin conductance, showed strong associations with engagement, particularly under high-immersion conditions. Conclusion: The study successfully identified neurophysiological markers of sense of presence and preliminarily validated the $\text{SoP}_{\text{MI}}$ index as a potential objective measure for VR applications. These findings establish a foundation for reliable and immersive VR experiences across fields, including training, rehabilitation and industry 5.0. Significance: By providing an objective and multimodal framework for measuring sense of presence, this research contributes to advancing VR applications where the sense of presence accurate and reliable assessment is essential. The $\text{SoP}_{\text{MI}}$ index offers potential for enhancing VR design and creating more effective, user-centered immersive experiences.