Developing an Adaptive AI Agent using Supervised and Reinforcement Learning with Monte Carlo Tree Search in FightingICE

Abstract

Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) are efficient algorithms for video game artificial intelligence (AI) agents, while Supervised Learning (SL) would make a video game AI agent visual-based. Combining SL and RL with MCTS has been tested in Computer Go, but this has yet to be thoroughly explored for fighting games. FightingICE, a 2D fighting game, serves as an ideal testing environment because of its complex action and observation spaces. In this paper, we use a Convolutional Neural Network (CNN) and Deep Q-Learning with MCTS (DQCN with MCTS) to create three models for FightingICE and compare their performance with an MCTS agent when playing against the same set of human testers. Our best performing model achieved a 58.57% game win-rate in 70 testing games after 7 training games. Although the model did not beat the MCTS agent's performance, it demonstrates the potential of combining SL, RL, and MCTS to develop an AI agent for fighting games.