SpaceGym: Discrete and Differential Games in Non-Cooperative Space Operations

Abstract

This paper introduces a collection of non-cooperative game environments that are intended to spur development and act as proving grounds for autonomous and AI decision-makers in the orbital domain. SpaceGym comprises two distinct suites of game environments: OrbitDefender2D (OD2D) and the Ker-bal Space Program Differential Games suite (KSPDG). OrbitDe-fender2D consists of discrete, chess-like, two-player gridworld games. OD2D game mechanics are loosely based on orbital motion and players compete to maintain control of orbital slots. The KSPDG suite consists of multi-agent pursuit-evasion differential games constructed within the Kerbal Space Program (KSP) game engine. In comparison to the very limited set of comparable environments in the existing literature, KSPDG represents a much more configurable, extensible, and higher-fidelity aerospace environment suite that leverages a mature game engine to incorporate physics models for phenomenon such as collision mechanics, kinematic chains for deformable bodies, atmospheric drag, variable-mass propulsion, solar irradiance, and thermal models. Both the discrete and differential game suites are built with standardized input/output interfaces based on OpenAI Gym and PettingZoo specifications. This standardization enables-but does not enforce-the use of rein-forcement learning agents within the SpaceGym environments. As a comparison point for future research, we provide baseline agents that employ techniques of model predictive control, numerical differential game solvers, and reinforcement learning-along with their respective performance metrics-for a subset of the SpaceGym environments. The SpaceGym software libraries can be found at https://github.com/mit-II/spacegym-od2d and https://github.com/mit-II/spacegym-kspdg.