Two-Step Deep Reinforcement Q-Learning based Relay Selection in Cooperative WPCNs

Abstract

In this paper, we propose an intelligent relay selection scheme employing deep reinforcement learning for a wireless powered cooperative network. We formulate the given problem as a Markov decision process with an unknown transitional probability between states. Therefore, a model-free off-policy relay selection model is proposed. The given model was deployed using a deep Q-network, with an updated relay selection process. Using channel characteristics, we find inaccessible nodes to form a pool of relays available for transmission and encourage the neural network to choose them. In addition, we propose a novel reward policy to train the model that is based on stored energy levels on the relays and promotes the system to expend energy. We numerically quantity the network performance in terms of outage probability and energy outage probability and compare them with the basic Q-learning.