Enhancing collaboration in multi-agent reinforcement learning with correlated trajectories

Abstract

Collaborative behaviors in human social activities can be modeled with multi-agent reinforcement learning and used to train the collaborative policies of agents to achieve efficient cooperation. In general, agents with similar behaviors have a certain behavioral common cognition and are more likely to understand the intentions of both parties then to form cooperative policies. Traditional approaches focus on the collaborative allocation process between agents, ignoring the effects of similar behaviors and common cognition characteristics in collaborative interactions. In order to better establish collaborative relationships between agents, we propose a novel multi-agent reinforcement learning collaborative algorithm based on the similarity of agents’ behavioral features. In this model, the interactions of agents are established as a graph neural network. Specifically, the Pearson correlation coefficient is proposed to compute the similarity of the history trajectories of the agents as a means of determining their behavioral common cognition, which is used to establish the weights of the edges in the modeled graph neural network. In addition, we design a transformer-encoder structured state information complementation module to enhance the decision representation of the agents. The experimental results on Predator–Prey and StarCraft II show that the proposed method can effectively enhance the collaborative behaviors between agents and improve the training efficiency of collaborative models.