Optimizing Performance of Worst Case User in Ultra-Dense Networks utilizing Deep Q-learning

In Ultra-Dense Networks (UDNs), where the Base Stations are distributed with a very high density, the users are possibly near the cells’ intersection. These users are called the Worst-Case Users (WCU) and usually experience very low performance Thus, improving the WCU performance is an urgent problem to secure the service requirement of future cellular networks. In this paper, the performance of the WCU is analyzed in UDNs with a maximum power algorithm and under the wireless environment with Stretched Path Loss model and Rayleigh fading. To improve the WCU data rate, the Deep Q Networks with and without Multi-Input-Multi-output (MIMO) are utilized in this paper. The simulation results show that a system–based Deep Q Learning can dramatically improve the WCU performance compared to the system with the maximum power algorithm. In addition, the deployment of the MIMO technique in a system–based Deep Q-learning only has benefits in bad channel conditions. In any channel condition, utilization of Deep Q Learning is a suitable solution to improve the WCU performance. Furthermore, if the user experiences a good channel condition, the MIMO technique can be used with Deep Q Learning to obtain further performance improvement.