Efficient Initial Access Based on DRL-Empowered Beam Sweeping

Abstract

Initial access (IA) is a procedure of establishing an initial connection between the base station (BS) and the users. In the fifth generation (5G) mobile communication system, the IA procedure includes beam management, which determines the beam pairs for random access (RA) and data transmission by beam sweeping. The existing beam sweeping method in the 3-rd generation partnership project (3GPP) standard mainly uses a predefined uniform beamforming codebook and sweeps the beams progressively, which is time-consuming and highly inflexible. In this paper, inspired by the fact that the highly non-uniform environment and user distribution mean part of the beam sweeping might be less beneficial, we propose a novel learning-based IA framework for the BS to optimize the beam sweeping patterns. Specifically, we resort to the deep reinforcement learning (DRL) approach to implicitly obtain the unknown environment and user distribution properties by continuously interacting with the environment, and then make decisions based on the rewards achieved by past actions. The simulation results show that our proposed scheme can save much time compared with the new radio (NR) and optimization methods under different datasets and conditions, which greatly improves the beam sweeping efficiency.