Nash-Minmax Strategy for Multiplayer Multiagent Graphical Games With Reinforcement Learning

Abstract

In this article, we address the synchronization problem in multiplayer multiagent graphical games, where each agent has multiple control input players. Herein, an agent represents a system, and the agent's control input represents a player's outcome. We formulate a Nash-minmax strategy, where the interactions of players in the same agent are nonzero-sum, while interactions of players between agents are antagonistic (e.g., zero-sum game). That is, the players in each agent minimize their costs, while the players from neighboring agents go against and maximize the costs. This approach finds the Nash control solutions for players within each agent and the worst control solutions for players in neighboring agents. The asymptotic stability under mild conditions and Nash-minmax solutions are guaranteed in the games. Offline policy iteration and online data-driven off-policy reinforcement learning algorithms are proposed, with proven convergence, to compute the Nash-minmax solutions. A simulation example validates the proposed strategy and algorithms.