Multi-Agent Coordination Profiles through State Space Perturbations

Abstract

The current work utilized a multi-agent reinforcement learning (MARL) algorithm embedded in a continuous predator-prey pursuit simulation environment to measure and evaluate coordination between cooperating agents. In this simulation environment, it is generally assumed that successful performance for cooperative agents necessarily results in the emergence of coordination, but a clear quantitative demonstration of coordination in this environment still does not exist. The current work focuses on 1) detecting emergent coordination between cooperating agents in a multi-agent predator-prey simulation environment, and 2) showing coordination profiles between cooperating agents extracted from systematic state perturbations. This work introduces a method for detecting and comparing the typically 'black-box' behavioral solutions that result from emergent coordination in multi-agent learning spatial tasks with a shared goal. Comparing coordination profiles can provide insights into overlapping patterns that define how agents learn to interact in cooperative multi-agent environments. Similarly, this approach provides an avenue for measuring and training agents to coordinate with humans. In this way, the present work looks towards understanding and creating artificial team-mates that will strive to coordinate optimally.