Fuzzy and Tile Coding Function Approximation in Agent Coevolution

Reinforcement learning (RL) is a machine learning technique for sequential decision making. This approach is well proven in many small-scale domains. The true potential of this technique cannot be fully realised until it can adequately deal with the large domain sizes that typically describe real world problems. RL with function approximation is one method of dealing with the domain size problem. This paper investigates two different function approximation approaches to RL: Fuzzy Sarsa and gradient descent Sarsa(λ) with tile coding. It presents detailed experiments in two different simulation environments on the effectiveness of the two approaches. Initial experiments indicated that the tile coding approach had greater modelling capabilities in both testbed domains. However, experimentation in a coevolutionary scenario has indicated that Fuzzy Sarsa has greater flexibility.