Cooperation With Humans of Unknown Intentions in Confined Spaces Using the Stackelberg Friend-or-Foe Game

Abstract

In confined indoor spaces, effectively modeling the cooperative behavior of uncrewed aerial vehicles (UAVs) with humans is critical to avoid moving obstacles and resolve deadlock situations. However, the unpredictable nature of the moving obstacles and uncertainty of human intentions pose significant challenges to autonomous navigation. In this work, we address this issue by formulating the problem as the “Imagined Friend-or-Foe Game,” where the UAV considers humans as friends and moving obstacles as foes. We introduce the Stackelberg friend-or-foe multiagent deep deterministic policy gradient algorithm to mitigate cycling, accelerate convergence, and enhance performance through the information advantage. Built upon the multiagent deep deterministic policy gradient framework, the proposed end-to-end learning architecture with reward shaping enables the UAV to cooperate with humans of unknown intentions based on local information. We empirically evaluate our proposed algorithm and architecture in narrow indoor scenarios, demonstrating that the Stackelberg friend-or-foe deep deterministic policy gradient algorithm improves deadlock relief and outperforms baseline algorithms.