A soft actor–critic reinforcement learning approach for over the air active beamforming with reconfigurable intelligent surface

Abstract

Combining reconfigurable intelligent surface (RIS) and unmanned aerial vehicle (UAV) provides ubiquitous connectivity for 6G air–ground communications, effectively enhancing coverage. However, due to the ”multiplicative fading” effect, passive RIS can only offer weak capacity gain. In addition, the mobility of UAVs may lead to imperfect channel state information (CSI), making it difficult to perform accurate beamforming. To address these issues, this paper adopts active RIS to actively amplify the reflected signals to overcome the high path loss caused by ”multiplicative fading”. To adapt to the randomness of channel changes, this paper employs the soft actor–critic (SAC) algorithm, which is based on the maximum entropy strategy. This approach jointly optimizes the precoding of the base station (BS) and the beamforming of the aerial RIS (ARIS), aiming to maximize the multi-user transmission rate. Simulation results show that when active ARIS is employed, the proposed algorithm achieves similar sum-rate results in imperfect CSI and perfect CSI scenarios and realizes 71% and 74% performance improvement compared to the traditional passive RIS, respectively. Moreover, the sum-rate remains stable within a certain range when the UAV hovers at any position between the BS and the user.