Hard-deadline-based frame filtering mechanism supporting the delivery of real-time video streams

Abstract

This paper describes a cross-layer filtering mechanism which facilitates real-time video frames to meet their stringent decoding deadlines in the existence of network congestion. The basic idea is to remove the dysfunctional video frames, which have missed their decoding deadlines, from transmission as early as possible, since they no longer serve for the functioning of a real-time media streaming application. The filtering mechanism consists of a pair of components which operate at the encoder and the decoder, respectively. The decoder-side component identifies the dysfunctional frames and sends the notifications to the encoder. The encoder-side component removes the identified dysfunctional frames from transmission. By removing dysfunctional frames from transmission, the video frames that are behind the dysfunctional frames are eligible for transmission at an earlier time and are made likely to meet their decoding deadlines. Meanwhile, removing dysfunctional frames from transmission also serves to maintain a stable and low queueing delay. The filtering mechanism relies on a user-space transport stack which enables the application-controlled transmission of data segments. The effectiveness of the filtering mechanism has been demonstrated through experiments in emulated networks.