A Conflict Rainbow DQN-Based Two-Stage Optimization Framework for Multiple Agile Satellites Scheduling

Abstract

With the rapid advancement of space technology and the increasing number of orbital satellites, the diverse task requirements are increasingly complex, and the satellite system is exhibiting trends toward intelligence and informatization. This article articulates a conflict rainbow deep Q network (DQN)-based two-stage scheduling framework that integrates the perspicacity of reinforcement learning with the cornerstone methodologies of conventional satellite task scheduling, engendering efficiency in task execution. In the prior stage, our framework employs a conflict-aware algorithm that meticulously analyzes and resolves resource conflicts based on the spatiotemporal characteristics of task visibility windows. This stage sets a robust foundation for the rear stage by efficiently allocating tasks to appropriate resources with minimized conflicts. Subsequently, the Rainbow DQN focuses on the coordination and implementation of task scheduling. Optimize scheduling decisions based on operational data and environmental changes to ensure task accomplishment under operational constraints. Experimental results show that the new proposed two-stage optimization framework demonstrates satisfactory execution efficiency over different task types and scales, while also delivering robust task execution outcomes.