Collaborative Content Caching Algorithm for Large-Scale ISTNs Based on MAPPO

Abstract

In this letter, we investigate the content caching problem within large-scale integrated satellite-terrestrial networks, focusing on a fusion scenario of future large-scale remote sensing constellations and communication satellite networks. Our investigation relies on deep reinforcement learning techniques aimed at minimizing the long-term average content delivery delay. To address the inherent challenge of convergence in single-agent algorithms, we propose clustering intelligent remote sensing satellites, with each cluster headed by an intelligent agent. Based on the characteristics of the model, we modify the multi-agent proximal policy optimization (MAPPO) algorithm by integrating long short-term memory (LSTM) to capture the correlation of the state information of different agents in the time domain. Simulation results show that the proposed LSTM-MAPPO outperforms the benchmarks, exhibiting faster convergence speed and lower standard deviation.