Enhancing Decentralized Federated Learning With Model Pruning and Adaptive Communication

Abstract

Federated learning (FL) is a distributed learning paradigm that enables large-scale IoT devices to collaboratively train a shared model while preserving the privacy of local data. To avoid the single-point-of-failure of the conventional parameter server architecture, the study concentrates on the decentralized FL (DFL) paradigm building on the device-to-device communication network. However, existing DFL frameworks encounter challenges related to resource limitations, privacy protection, and data heterogeneity. To overcome these challenges, the study proposes and implements DF$^{2}$-MPC in industrial IoT, an efficient DFL framework with personalized model pruning and adaptive communication. Specifically, a personalized pruning ratio determination approach is designed by exploiting the model pruning technique. This approach enables all devices to flexibly determine pruning ratios by themselves, thereby achieving both communication savings and privacy protection. Then, this study designs an adaptive neighbor selection scheme, which can enhance model performance and foster model consensus under resource constraints. In addition, the study theoretically proves the convergence performance of DF$^{2}$-MPC. Finally, extensive simulations on three real-world traces are conducted to corroborate the superiority of DF$^{2}$-MPC, demonstrating that the method can improve communication efficiency with satisfactory model accuracy and convergence performance.