Extension of Improved Penalty Avoiding Rational Policy Making algorithm to tile coding environment for keepaway tasks

Abstract

We focus on potential capability of a profit sharing method (PS) in non-Markov multi-agent environments. It is shown that PS has some rationality in non-Markov environments and is also effective in multi-agent environments. However, conventional PS uses only a reward to learn suitable rules. On the other hand. ldquopenalty avoiding rational policy making algorithm (PARP)rdquo is based on PS and uses not only a reward but also penalties. PARP is improved to save memories and to cope with uncertainties, which is known as ldquoimproved penalty avoiding rational policy making algorithm (improved PARP).rdquo There is another critical problem we must cope with when we apply PS based methods to real environments; we need a huge amount of state information and most of states take continuous values. One solution for this problem is to approximate the states with a function approximation method, e.g. tile coding. In this paper, first, we extend improved penalty avoiding rational policy making algorithm to tile coding environments. Then, we compare the extended method with conventional methods to show the effectiveness through an application to a keepaway task in a soccer game.