Cooperative Behavior of Agents That Model the Other and the Self in Noisy Iterated Prisoners' Dilemma Simulation

Abstract

We developed self learning for simulation study of mutual understanding between peer agents. We designed them to use various types of coplayer models and a reinforcement learning algorithm to learn to play a noisy iterated prisoners' dilemma game so that the pay-off for the agent itself is maximized. We measured the mutual-modeling ability of each type of agent in terms of cooperative behavior when playing with another equivalent agent. We observed that agents with a complex coplayer model, which includes a model of the agent itself, showed higher cooperation than agents with a simpler coplayer model only. Moreover, in low-noise environments, Level-M agent, which develops equivalent models of the self and the other, showed higher cooperation than other types of agents. These results suggest the importance of "self-observation" in the design of communicative agents