Privacy-Preserving Gaze-Assisted Immersive Video Streaming

Abstract

Immersive videos, also known as 360 $^{\circ }$ videos, have gained significant attention in recent years due to their ability to provide an interactive and engaging experience. However, the development of immersive video streaming faces several challenges, including privacy concerns, the need for accurate viewport prediction, and efficient bandwidth allocation. In this paper, we propose a comprehensive system that integrates three specialized modules: the Privacy Protection module, the Viewport Prediction module, and the Bitrate Allocation module. The Privacy Protection module introduces a novel approach to differential privacy tailored for immersive video environments, considering the spatial and temporal correlations in viewport and gaze motion data. The Viewport Prediction module leverages a crossmodal attention mechanism based on the transformer to predict user viewport movements by analyzing the complex interactions between historical data, video content, and gaze patterns. The Bitrate Allocation module employs an adaptive tile-based bitrate allocation strategy using an exponential decay function to optimize video quality and maximize user quality of experience. Experimental results demonstrate that our proposed framework outperforms three state-of-the-art integrated frameworks, achieving an average QoE improvement of 21.61%. This paper offers substantial novelty in addressing privacy concerns, leveraging gaze information for viewport prediction, and utilizing underlying correlations between different features.