DESIGN OF SELF-ORGANIZING SYSTEMS USING MULTI-AGENT REINFORCEMENT LEARNING AND THE COMPROMISE DECISION SUPPORT PROBLEM CONSTRUCT

Abstract

In this paper, we address the following question: How can multi-robot self-organizing systems be designed so that they show the desired behaviors and are able to perform tasks specified by designers? Multi-robot self-organizing systems, e.g., swarm robots, have great potential for adapting when performing complex tasks in a changing environment. However, such systems are difficult to design due to the stochasticity of the system performance and the non-linearity between the local actions/interaction and the desired global behavior. In order to address this, in this paper we propose a framework for designing self-organizing systems using Multi-Agent Reinforcement Learning (MARL) and the compromise Decision-Support Problem (cDSP) construct. In this paper we present a framework that consists of two stages, namely, preliminary design and design improvement. In the preliminary design stage, MARL is used to help designers train the robots so that they show stable group behavior for performing the task. In the design improvement stage, the cDSP construct is used to explore the design space and identify satisfactory solutions considering several performance indicators. Between the two stages, surrogate models are used to map the relationship between local parameters and global performance indicators utilizing the data generated in preliminary design. A multi-robot box-pushing problem is used as an example to test the efficacy of the framework. The framework is general and can be extended to design other self-organizing systems. Our focus in this paper is in describing the framework.