Multi Q-Table Q-Learning

Abstract

Q-learning is a popular reinforcement learning technique for solving shortest path (STP) problem. In a maze with multiple sub-tasks such as collecting treasures and avoiding traps, it has been observed that the Q-learning converges to the optimal path. However, the sum of obtained rewards along the path in average is moderate. This paper proposes Multi-Q-Table Q-learning to address a problem of low average sum of rewards. The proposed method constructs a new Q-table whenever a sub-goal is reached. This modification let an agent to learn that the sub-reward is already collect and it can be obtained only once. Our experimental results show that a modified algorithm can achieve an optimal answer to collect all treasures (positive rewards), avoid pit and reach goal with the shortest path. With a small size of maze, the proposed algorithm uses the larger amount of time to achieved optimal solution compared to the conventional Q-learning.