Deep reinforcement learning-based resource allocation method for multi-satellite scheduling

Abstract

Agile Earth observation satellites (AEOSs) scheduling represents a complex domain within combinatorial optimization, crucial for the regular operations and mission success of in-orbit satellites. In order to timelessly tackle the allocation of complex resources and corresponding time windows, a satellite resource adaptive allocation method, named SRADA-DRL, is proposed in this paper. By combining deep reinforcement learning (DRL) with rule-based heuristics, the SRADA-DRL is designed to optimize the allocation of satellite resources in dynamic environments. Concerning maximizing the total rewards of allocated missions, a mathematical model and a corresponding Markov decision model are constructed within the scheduling process. After analyzing the spatial–temporal distribution features of all resources and missions, the time-dependent missions are first decomposed into meta-missions corresponding to satellite resources, and a meta-mission is then selected to generate an allocation sequence in each stage. On this basis, the execution times for all missions are assigned in the single-satellite scheduling process. In which, the DRL updates the gradient information contingent upon the rewards garnered from the allocation sequence. In addition, the classical scheduling scenarios of varying scales are also conducted. Experimental results demonstrate the effectiveness and efficiency of the proposed SRADA-DRL method in addressing the AEOSs scheduling.