A Cooperation-Free Resource Allocation Algorithm Enhanced by Reinforcement Learning for Coexisting IIoTs

Abstract

The Industrial Internet of Things (IIoTs) plays an important role in various industrial applications, which require multiple time-critical networks to be deployed in the same region. The limited communication resources inevitably incur network coexistence problems. For scenarios where coexisting networks cannot coordinate effectively, the centralized or partial-information-based decentralized resource allocation methods cannot be implemented. To address this concern, we propose a Cooperation-Free Reinforcement Learning (CF-RL) algorithm for the fully distributed resource allocation problem in coexisting IIoT systems. Each network adopts the proposed algorithm to minimize collisions through a trial-and-error approach without any information interaction. To resist the influence of environmental dynamics, each coexisting network learns the state transition probability of the resource block instead of the resource block's position. Moreover, to potentially ensure the overall system performance, each network additionally considers the period offset in the initialization phase and action selection phase, so that the coexisting networks have different preferences for different state transitions. We conduct extensive simulations to verify the convergence performance. Evaluation results show that the CF-RL algorithm almost achieves (more than 99.88%) the effect of centralized resource allocation and has obvious superiorities over other cooperation-free algorithms in terms of the convergence rate, the number of collisions, and the resource utilization ratio.