A Categorization of Reinforcement Learning Exploration Techniques which Facilitates Combination of Different Methods

Abstract

The exploration vs. exploitation problem in reinforcement learning has received much attention lately. It is difficult to get a complete overview of all the exploration methods, and what methods can or can not be used together in a combined algorithm. We propose a categorization of exploration techniques based on the mechanism through which they generate exploratory policies. This enables plug-and-play combination of exploration techniques from each category. We show how to combine methods from different categories and demonstrate that such a combination can result in a new exploration technique which outperforms each individual method.