Model-Free Learning Algorithms for Dynamic Transmission Control in IoT Equipment

Abstract

We consider an energy-harvesting IoT device transmitting delay- and jitter-sensitive data over a wireless fading channel. Given the limited harvested energy, our goal is to compute optimal transmission control policies that decide on how many packets of data should be transmitted from the buffer's head-of-line at each discrete timeslot such that a long-run criterion involving the average delay/jitter is either minimized or never exceeds a pre-specified threshold. We utilize a suite of Q-learning-based techniques (from the reinforcement learning theory) to optimize the transmission policy in a model-free fashion. Compared to prior work, our novelty lies in proposing a model-free learning algorithm that enables jitter-aware transmissions by penalizing control decisions with the variance of the delay cost function. Extensive numerical results are presented for performance evaluation.