Reinforcement Learning for Robust Header Compression (ROHC) Under Model Uncertainty

Abstract

Robust header compression (ROHC), critically positioned between network and MAC layers, plays an important role in modern wireless communication networks for improving data efficiency. This work investigates bi-directional ROHC (BD-ROHC) integrated with a novel architecture of reinforcement learning (RL). We formulate a partially observable Markov decision process (POMDP), where the compressor is the POMDP agent, and the environment consists of the decompressor, channel, and header source. Our work adopts the well-known deep Q-network (DQN), which takes the history of actions and observations as inputs, and outputs the Q-values of corresponding actions. Compared with the ideal dynamic programming (DP) proposed in existing works, the newly proposed method is scalable to the state, action, and observation spaces. In contrast, DP often incurs formidable computation costs when the number of states becomes large due to long decompressor feedback delays and complex channel models. In addition, the new method does not require prior knowledge of the transition dynamics and accurate observation dependency of the model, which are often unavailable in practical applications.