Policy iteration vs Q-Sarsa approach optimization for embedded system communications with energy harvesting

In this paper, we consider a wireless point-to-point communication in the context of battery powered embedded systems with energy harvesting equipment. The successive actions taken by the transmitter constitutes the policy that it follows. In the first stage, we suppose a limited knowledge of the system behavior characterized by its probability transition matrix, and then use the policy iteration algorithm to find the optimal policy. In the second stage, we consider that such basic stochastic knowledge is not available at the transmitter, and consider the Q-Sarsa algorithm to find out optimal policies. The two approaches are first simulated and then compared.