A Self-Organizing Distributed Reinforcement Learning Algorithm to Achieve Fair Bandwidth Allocation for Priority-Based Bus Communication

Abstract

Due to the raising complexity in distributed embedded systems, a single designer will not be able to plan and organize the communication for such systems. Therefore, it will get more and more important to relieve the designer in that task. Our idea is a communication system that is capable to organize itself to satisfy predefined properties. In this paper, we want to solve the problem of establishing fair bandwidth sharing on priority-based buses by using simple local rules on the distributed system to avoid a single point of failure and cope with online system changes. Based on a game theoretical analysis, a multi-agent reinforcement learning algorithm is proposed that establishes fair bandwidth distribution. The main idea is to penalize nodes that claim too much bandwidth by the other nodes. We experimentally evaluated the algorithm with different parameter settings. The algorithm showed to converge to a fair solution in any experiment. This means the system is able to completely self-organize without global information for our assumptions. In addition, we could figure out that we can configure a trade-off between convergence speed and computation effort. We hope this is a small first step towards totally self-organizing real-time systems.