Efficient Encoding of Interactive Personalized Views Extracted from Immersive Video Content

Abstract

Traditional television limits people to a single viewpoint. However, with new technologies such as virtual reality glasses, the way in which people experience video will change. Instead of being limited to a single viewpoint, people will demand a more immersive experience that gives them a sense of being present in a sports stadium, a concert hall, or at other events. To satisfy these users, video such as 360-degree or panoramic video needs to be transported to their homes. Since these videos have an extremely high resolution, sending the entire video requires a high bandwidth capacity and also results in a high decoding complexity at the viewer. The traditional approach to this problem is to split the original video into tiles and only send the required tiles to the viewer. However, this approach still has a large bit rate overhead compared to sending only the required view. Therefore, we propose to send only a personalized view to each user. Since this paper focuses on reducing the computational cost of such a system, we accelerate the encoding of each personalized view based on coding information obtained from a pre-analysis on the entire ultra-high-resolution video. By doing this using the High Efficiency Video Coding Test Model (HM), the complexity of each individual encode of a personalized view is reduced by more than 96.5% compared to a full encode of the view. This acceleration results in a bit rate overhead of at most 19.5%, which is smaller compared to the bit rate overhead of the tile-based method.