Joint Beamforming in RIS-Assisted Multi-User Transmission Design: A Model-Driven Deep Reinforcement Learning Framework

Abstract

The deployment of multiple reconfigurable intelligent surfaces (RIS) is a promising strategy to enhance wireless system performance. However, joint beamforming in multi-RIS assisted systems faces significant challenges due to the increased number of optimization variables, non-convex objective functions, and constraints. In this study, we propose an algorithm based on weighted minimum mean square error optimization and the successive convex approximation algorithm, maximizing the weighted sum rate in a double-RIS assisted downlink multi-user multiple-input single-output system. We also present a general framework for model-driven deep learning that addresses the limitations of existing methods, which often lack flexibility to different channels and suffer from a large training burden due to the high-dimensional action space of deep reinforcement learning (DRL). Initially, we configure the step size in the proposed algorithm as trainable, accelerating convergence. Then, a recurrent neural network generates the step size for iterations, allowing dynamic iteration extension in varying environmental conditions. We enhance the neural network’s self-adaptability by introducing a model-driven DRL algorithm, integrating expert knowledge into the DRL actor network’s design. Simulation results demonstrate up to 30% performance improvement over traditional algorithms, achieved by our model-driven framework. The proposed model-driven DRL shows higher capacity for dynamic extension and rapid adaptation to new environments.