Accelerating Deep Reinforcement Learning Under the Guidance of Adaptive Fuzzy Logic Rules

Abstract

While Deep Reinforcement Learning (DRL) has emerged as a prospective method to many tough tasks, it remains laborious to train DRL agents with a handful of data collection and high sample efficiency. In this paper, we present an Adaptive Fuzzy Reinforcement Learning framework (AFuRL) for accelerating the learning process by incorporating adaptive fuzzy logic rules, enabling DRL agents to improve the efficiency of exploring the state space. In AFuRL, the DRL agent first leverages prior fuzzy logic rules designed especially for the actor-critic framework to learn some near-optimal policies, then further improves these policies by automatically generating adaptive fuzzy rules from state-action pairs. Ultimately, the RL algorithm is applied to refine the rough policy obtained by a fuzzy controller. We demonstrate the validity of AFuRL in both discrete and continuous control tasks, where our method surpasses DRL algorithms by a substantial margin. The experiment results show that AFuRL can find superior policies in comparison with imitation-based and some prior knowledge-based approaches.