Model-Driven Bayesian Reinforcement Learning for IRS-Assisted Massive MIMO-OFDM Channel Feedback, Beamforming, and IRS Control

Abstract

In Intelligent Reflecting Surface (IRS)-assisted massive Multiple-Input Multiple-Output (MIMO) systems, the downlink channel state information (CSI) needs to be fed back to the base station (BS) and utilized to perform the beamforming and IRS control for high spectral efficiency performance. However, the intricate nature of these systems, characterized by a vast number of antennas, subcarriers, and IRS elements, exacerbates the CSI feedback overhead and complicates the optimization of beamforming and IRS parameters, potentially compromising spectral efficiency. Addressing these challenges, this paper introduces a Bayesian Reinforcement Learning (BRL)-based approach, named IRS-CSI-BRL, for efficient CSI feedback, beamforming, and IRS control. Firstly, the IRS-CSI-BRL approach utilizes the equivalent CSI for optimization, aligning with current channel estimation protocols without necessitating extensive modifications. Secondly, it employs a practical IRS control model that optimizes the effective capacitance of IRS control circuits rather than IRS reflection coefficients, accurately reflecting the IRS’s frequency-responsive behavior to enhance system performance. Additionally, we advocate bypassing the reconstruction of the CSI at the BS to eliminate information irrelevant to beamforming and IRS control, thereby boosting feedback efficiency. Another distinctive feature of the proposed scheme is that its output format is probability distributions, which enables the incorporation of model-assisted knowledge about the latent space and boosts the algorithm’s robustness. Simulation results demonstrate that the proposed IRS-CSI-BRL scheme significantly outperforms start-of-the-art solutions in feedback overhead reduction and system data rate enhancement while maintaining exceptional robustness. Furthermore, this approach maintains flexibility, allowing for the incorporation of an additional training loss function for full CSI reconstruction if needed.