A Scalable Game Theoretic Approach for Coordination of Multiple Dynamic Systems

Abstract

Learning in games provides a powerful framework to design control policies for self-interested agents that may be coupled through their dynamics, costs, or constraints. We consider the case where the dynamics of the coupled system can be modeled as a Markov potential game. In this case, distributed learning ensures agents’ control policies converge to a Nash equilibrium. However, standard algorithms like natural policy gradient require global state and action knowledge, which does not scale well with more agents. We show that by limiting information flow to local neighborhoods, we can still converge to near-optimal policies. If a game’s global cost function can be decomposed into local costs that align with agent policies at equilibrium, this approach benefits team coordination. We demonstrate this with a sensor coverage problem.