Utilization of Webots and Khepera II as a platform for Neural Q-Learning controllers

The Webots commercial mobile robot simulation software and Khepera II miniature mobile robot have always been popular tools in research centers and universities. In this paper, the two items will be utilized as a platform for the investigation of Neural Q-Learning controllers. Webots remains as the primary simulation software where the simulated environment and robot are modeled. To cater for a wide variety of experiments, the simulation developed for the Khepera II is equipped with GUIs and various features. These functions allow the user to configure different environment and robot settings for different experiments. Then, the simulation is validated by comparing the behavior of the simulated and actual robot. As a result, a total of four controllers is proposed and tested on this platform. The designed controllers include both sensor and vision based controllers. These controllers are capable of exhibiting obstacle avoidance or wall following behaviors. In addition, an obstacle avoidance controller which is based on a combination of sensor and visual inputs via a fuzzy logic controller was proposed. Experimental results collected facilitate comparison and discussion of the algorithm and it further reveals that the mobile robot could successfully acquire the desired behavior.