Studies on the less-used actions exploration problem of a rationing algorithm based on reinforcement learning

Abstract

Programming by demonstration is an interesting subject in the field of robotics and it is developing more and more in the direction of robots for services and humanoid robots. Programming by demonstration is much less researched when it comes to industrial robots. One of the reasons is that an industrial robot has to act in a precise and certain manner. However, extending research regarding programming by demonstration to the field of industrial robots could lead to the creation of intelligent systems where the industrial robot could be programmed in an easier way. The goal of our research is to develop an intelligent system useful for industrial robot programming by demonstration. The reasoning algorithms are the mechanisms which offer flexibility to the proposed system. We have focused our research on the creation of a reasoning algorithm based on artificial neural networks [1, 2]. Because the results of this algorithm were not satisfying we have switched our focus to the development of a reasoning algorithm based on reinforcement learning [3]. The algorithm is based on the idea that marks can be assigned to each possible action whenever the robot is in an unknown state. The exploration of less-used actions plays also an important role in the case the robot must to take a decision. Based on the marks and on the exploration feature of the algorithm the robot updates its behaviour. This paper presents a description and some studies on less-used actions exploration problem of the algorithm. Some chapters of the paper will deal with the problems implementing the algorithm, the conducted experiments in terms of exploration feature of the algorithm and the results obtained. The analysis of the results and the characteristics of the algorithm in terms of less-used actions exploration are also discussed in this paper.