Autonomous Command and Control for Earth-Observing Satellites using Deep Reinforcement Learning

Abstract

Autonomy is a highly sought-after feature for future civil, commercial, and military Earth-observation space missions. Deep Reinforcement Learning (DRL) techniques have demonstrated state-of-the-art performance for on-line decision making in complex domains such as competitive real-time strat-egy games and black-box control. DRL allows for the direct uti-lization of high-fidelity whole-system simulators without inter-mediate approximations or representations. As a result, agents trained with DRL can explore and exploit subtle dynamics that may be lost when applying approximation-driven techniques for tasking and planning, thereby enabling greater performance. This work describes efforts at Ball Aerospace in developing a DRL-driven solution to the single-satellite, arbitrary-target, single-ground-station planning problem. A design reference mission for this problem, based on the Compact Infrared Ra-diometer in Space (CIRiS) cubesat, is described alongside an im-plementation of this mission using the Basilisk simulation frame-work. The resulting simulator is used as a training environment for the DRL-based agent. This simulator is also used for performance evaluation. The DRL-based agent's performance is compared against the results of a rule-based agent. The results of these approaches are compared using multiple figures of merit, including objective performance, decision-making time, and mission-level resource utilization. The resulting comparison demonstrates the relative merits of DRL as an approach versus heuristic, rule-based command and control architectures.