Diffusion-Based Deep Reinforcement Learning for Resource Management in Connected Construction Equipment Networks: A Hierarchical Framework

Abstract

With the extensive adoption of information technology, tunnel construction is experiencing a rapid digital transformation. Integrating powerful direct communication among construction equipment (CE) facilitates real-time data exchange, promoting collaborative operations among CE. Concurrent execution of multiple construction procedures leads to a significant rise in the amount of CE and communication links, resulting in resource competition. However, this competition is aimed at enhancing collaboration. To address this inherently contradictory issue, we propose a hierarchical resource management framework and align communication quality of service (QoS) to construction efficiency using construction procedure coherence degree (CPCD) based on age of information (AoI). By formulating resource management as a stochastic optimization problem, a suitable online two-level deep reinforcement learning algorithm referred to as diffusion based soft actor critic (DSAC)-QMIX is designed to derive the radio resource allocation strategies. DSAC is responsible for orchestrating spectrum inter-fleets at the high-level, and QMIX makes the resource management and power control decision for each CE at the low-level. Simulation results validate the effectiveness of the DSAC-QMIX algorithm with comparable transmission rate, and show superior performance in terms of CPCD satisfaction compared with other benchmarks.