QSFL: Two-Level Communication-Efficient Federated Learning on Mobile Edge Devices

Abstract

In cross-device horizontal federated learning (FL), the communication cost of transmitting complete models between edge devices and a central server is a significant bottleneck, due to expensive, unreliable, and low-bandwidth wireless connections. As a solution, we propose a novel FL framework named QSFL , towards optimizing FL uplink (client-to-server) communication at both client and model levels . At the client level, we design a Qualification Judgment (QJ) algorithm to sample high-qualification clients to upload models. At the model level, we design a Sparse Cyclic Sliding Segmentation (SCSS) algorithm to further compress the local model transmitted from the client to the server in the uplink communication. We prove that QSFL can converge over wall-to-wall time, and develop an optimal hyperparameter searching algorithm based on theoretical analysis to enable QSFL to make the best trade-off between model accuracy and communication cost. Experimental results show that QSFL achieves state-of-the-art compression ratios with marginal model accuracy degradation. Since mobile edge devices as FL clients often have heterogeneous system resources, such as communication bandwidth, we propose two novel dynamic segmentation strategies with varied counts or sizes based on QSFL to enhance the robustness of QSFL to FL system heterogeneity. For some mobile edge devices joining as FL clients with both limited uplink and downlink communication bandwidths, they can not pull up the global model from the server. To tackle it, we propose a novel symmetric downlink compression scheme on top of QSFL to further reduce the downlink (server-to-client) communication costs, hence enabling a bidirectional communication-efficient FL. Theory analysis and experiments demonstrate that QSFL with dynamic segmentation or symmetric downlink compression still keeps convergence and takes a better trade-off between model accuracy and communication efficiency than without them.