Learning Strategies for Imperfect Information Board Games Using Depth-Limited Counterfactual Regret Minimization and Belief State

Abstract

Counterfactual Regret Minimization (CFR) variants have mastered many Poker games by effectively handling a large number of opportunities in private information within relatively short playing histories of the game. However, for imperfect information board games with infrequent chance events but long histories or even loops, the effectiveness of CFR is often limited in practice as the computational complexity grows exponentially with the game length. In this paper, we propose Belief States with Approximation by Dirichlet Distributions and Depth-limited External Sampling for Board Games that enables an effective abstraction even with existence of loops. Experiments show that our proposed methods have the ability to learn reasonable strategies.