Communication-Efficient Federated Learning Over Capacity-Limited Wireless Networks

Abstract

In this paper, we propose a communication-efficient federated learning (FL) framework to enhance the convergence rate of FL under limited uplink capacity. The core idea of our framework is to transmit the values and positions of the top-S entries of a local model update, determined in terms of magnitude. When transmitting the top-S values, we first apply a linear transformation that enforces the transformed values to behave like Gaussian random variables. We then employ a scalar quantizer optimized for Gaussian distributions, leading to minimizing compression errors. When reconstructing the top-S values, we develop a linear minimum mean squared error method based on the Bussgang decomposition. Additionally, we introduce an error feedback strategy to compensate for both compression and reconstruction errors. We analyze the convergence rate of our framework under general considerations, including a non-convex loss function. Based on our analytical results, we optimize the key parameters of our framework to maximize the convergence rate for a given uplink capacity. Simulation results demonstrate that our framework achieves more than a 2.2%, 1.1%, and 1.4% increase in classification accuracy for the MNIST, CIFAR-10, and CIFAR-100 datasets, respectively, compared to state-of-the-art FL frameworks under limited uplink capacity.