Learning and evolving combat game controllers

Abstract

The design of the control mechanisms for the agents in modern video games is one of the main tasks involved in the game design process. Designing controllers grows in complexity as either the number of different game agents or the number of possible actions increase. An alternative mechanism to hard-coding agent controllers is the use of learning techniques. This paper introduces two new variants of a hybrid algorithm, named WEREWoLF and WERESARSA, that combine evolutionary techniques with reinforcement learning. Both new algorithms allow a group of different reinforcement learning controllers to be recombined in an iterative process that uses both evolution and learning. These new algorithms have been tested against different instances of predefined controllers on a one-on-one combat simulator, with underlying game mechanics similar to classic arcade games of this kind. The results have been compared with other reinforcement learning controllers, showing that WEREWoLF outperforms the other algorithms for a series of different learning conditions.