Dynamic Beam Hopping for DVB-S2X Satellite: A Multi-Objective Deep Reinforcement Learning Approach

Abstract

Dynamic Beam Hopping (DBH) is a crucial technology for adapting to the flexibility of different service configurations in the multi-beam satellite communications market. The conventional beam hopping method, which ignores the intrinsic correlation between decisions, only obtains the optimal solution at the current time, while deep reinforcement learning (DRL) is a typical algorithm for solving sequential decision problems. Therefore, to deal with the DBH problem in the scenario of Differentiated Services (DIFFSERV), this paper designs a multiobjective deep reinforcement learning (MO-DRL) algorithm. Besides, as the demand for the number of beams increases, the complexity of system implementation increase significantly. This paper innovatively proposes a time division multi-action selectionmethod(TD-MASM) tosolvethecurseofdimensionality problem. Under the real condition, the MO-DRL algorithm with the low complexity can ensure the fairness of each cell, improve the throughput to about 5540Mbps, and reduce the delay to about 0.367ms. The simulation results show that when the GA is used to achieve similar effects, the complexity of GA is about 110 times that of the MO-DRL algorithm.