Convergence Improvement of Q-learning Based on a Personalized Recommendation System

Abstract

Reinforcement Learning (RL) – a research branch of artificial intelligence – exploits the concept of reward/penalty from human learning so that the feedback information of the environment can be used for self-learning without previous knowledge. Although RL has been applied to many research fields, there are difficulties when implementing it in some real world applications. One difficulty is the tradeoff between exploration and exploitation for the agent of RL in choosing a proper action. An improper action may lead to a learning failure or an increase in learning cost. Another difficulty is that the learning agent of RL needs to interact with the environment to attain a real-time reward/penalty; however, the learning time spent in the interaction process may be too long. In order to overcome the aforementioned difficulties, this paper proposes an approach that employs a personalized recommendation system to provide a feedforward candidate action for RL to implement self-adaptive learning through teaching. A real visual tracking experiment using a pan-tilt camera system is conducted to assess the performance of the proposed approach. Experimental results show that the proposed personalized recommendation system-based approach is able to improve the effectiveness and practicality of RL.