Multi - Agent Deep Deterministic Policy Gradient Based Satellite Spectrum/Code Resource Scheduling with Multi-constraint

Abstract

For multi-user satellite Internet of Things (IoT) systems operating at lower signal-to-noise ratio, spread spectrum techniques are usually used to combat narrowband interference. In addition, the communication performance in the spread spectrum system depends on the anti-jamming ability of the spreading codes (SCs). Therefore, how to design the SCs distributed scheduling strategies under multi-users requirements and resource constraints has become a crucial problem for satellite IoT systems. In this paper, the number of collisions and the amount of transmitted data are introduced as gauges to measure the distributed scheduling performance of the satellite multi-user systems. Specifically, terminal gateways (TGs) must efficiently and effectively select limited available SCs according to their state at each communication time slot independently. The SCs distributed scheduling problem is formulated as a Markov Decision Process (MDP) along with the observed environments composed of resource status and TGs status. Then a deep rein-forcement learning scheduling algorithm is devised by combining the A2C framework and the idea of multi-user. Simulation results show that the proposed algorithm can achieve much better performance than traditional algorithms in reducing scheduling conflicts and improving communication efficiency. Finally, we draw some conclusions.