Depth camera assisted multiterminal video coding

Abstract

This paper addresses multiterminal video coding with the help of a low-resolution depth camera. In this setup, the depth sequence, together with the high-resolution texture sequences, are collected, compressed and transmitted separately to the joint decoder in order to obtain more accurate depth information and consequently better rate-distortion performance. At the decoder end, side information (for Wyner-Ziv coding) is generated based on successive refinement of the decompressed low-resolution depth map and texture frame warped from other terminals. Experimental results show sum-rate savings with the depth camera than without for the same PSNR performance. Comparisons to simulcast and JMVM coding are also provided. Although the sum-rate gain of our multiterminal video coding scheme (with or without the depth camera) over simulcast is relatively small, this work is the first that incorporates depth camera in a multiterminal setting.