Multi-agent collaborative operation planning via cross-domain transfer learning

Abstract

Transfer learning has shown promising potentials in assisting multi-agent systems (MAS) to deal with complex collaborative tasks. In this work, we investigate MAS collaboration in 3D underwater environment. In response to the problem of high sampling cost in underwater operation when multi-agent without any prior knowledge, the multi-agent collaborative operation planning via cross-domain transfer learning (CDTL) is proposed. In CDTL, the training process of MAS is accelerated through learning the domain invariant knowledge from the samples of 2D ground collaborative tasks that easily obtained. First, the samples in ground tasks are divided into six state phases based on the semantic order of task execution, and a state transition graph is constructed accordingly. Then, a domain adaptation method with inter-class relationship (ICDA) is proposed, which focuses on the invariant semantic structure of the ground (source) and the underwater (target) task to capture prior knowledge. During the knowledge transferring, ICDA is used to correct decision of the agents’ policies that based on MAX-Q controller. Finally, the extensive experiments show that CDTL reduces the cost of physical time by 37.3% when the MAS completes the new task for the first time.