Improving Temporal Difference game agent control using a dynamic exploration during control learning

Abstract

This paper investigates the use of a dynamically generated exploration rate when using a reinforcement learning-based game agent controller within a dynamic digital game environment. Temporal Difference learning has been employed for the real-time gereration of reactive game agent behaviors within a variation of classic arcade game Pac-Man. Due to the dynamic nature of the game environment initial experiments made use of static, low value for the exploration rate utilized by action selection during learning. However, further experiments were conducted which dynamically generated a value for the exploration rate prior to learning using a genetic algorithm. Results obtained have shown that an improvement in the overall performance of the game agent controller may be achieved when a dynamic exploration rate is used. In particular, if the use of the genetic algorithm is controlled by a measure of the current performance of the game agent, further gains in the overall performance of the game agent may be achieved.