Impact of Task Constraint on Agent Team Size of Self-Organizing Systems Measured by Effective Entropy

Abstract

Self-organizing systems can perform complex tasks in unpredictable situations with adaptability. Previous work has introduced a multiagent reinforcement learning based model as a design approach to solving the rule generation problem with complex tasks. A deep multiagent reinforcement learning algorithm was devised to train self-organizing agents for knowledge acquisition of the task field and social rules. The results showed that there is an optimal number of agents that achieve good learning stability and system performance. However, finding such a number is nontrivial due to the dynamic task constraints and unavailability of agent knowledge before training. Although extensive training can eventually reveal the optimal number, it requires training simulations of all agent numbers under consideration, which can be computationally expensive and time-consuming. Thus, there remains the issue of how to predict such an optimal team size for self-organizing systems with minimal training experiments. In this paper, we proposed a measurement of the complexity of the self-organizing system called effective entropy, which considers the task constraints. A systematic approach, including several key concepts and steps, is proposed to calculate the effective entropy for given task environments, which is then illustrated and tested in a box-pushing case study. The results show that our proposed method and complexity measurement can accurately predict the optimal number of agents in self-organizing systems, and training simulations can be reduced by a factor of 10.