A fast strategy-solving method for adversarial team games utilizing warm starting

Abstract

Adversarial team games (ATGs) have garnered significant attention in recent years, leading to the emergence of various solutions such as linear programming algorithms, multi-agent reinforcement learning, and game tree transformation. The ATGs involve large-scale game trees, resulting in higher time costs for computation. In this paper, we focus on expediting the solution of team-maxmin equilibrium with correlation (TMECor), which can be considered the equilibrium that maximizes the team’s payoff. To address this, we propose a transformation with seed strategies (TSS). TSS leverages reinforcement learning to calculate player strategies. We initialize the strategies of all players, referred to as seed strategies, and incorporate them into the multi-agent game tree during the transformation process. These seed strategies serve as the starting strategies for counterfactual regret minimization (CFR). CFR initializes the strategies and cumulative regret of all players based on the seed strategy. By warm starting the whole process, our method accelerates the solving of TMECor. We conducted nine experiments using Kuhn poker and Leduc Hold’em poker. The results demonstrated that TSS improved the solving speed of TMECor.