P2CEFL: Privacy-Preserving and Communication Efficient Federated Learning With Sparse Gradient and Dithering Quantization

Abstract

Federated learning (FL) offers a promising framework for obtaining a global model by aggregating trained parameters from participating clients without transmitting their local private data. To further enhance privacy, differential privacy (DP)-based FL can be considered, wherein certain amounts of noise are added to the transmitting parameters, inevitably leading to a deterioration in communication efficiency. In this paper, we propose a novel Privacy-Preserving and Communication Efficient Federated Learning (P2CEFL) algorithm to reduce communication overhead under DP guarantee, utilizing sparse gradient and dithering quantization. Through gradient sparsification, the upload overhead for clients decreases considerably. Additionally, a subtractive dithering approach is employed to quantize sparse gradient, further reducing the bits for communication. We conduct theoretical analysis on privacy protection and convergence to verify the effectiveness of the proposed algorithm. Extensive numerical simulations show that the P2CEFL algorithm can achieve a similar level of model accuracy and significantly reduce communication costs compared to existing conventional DP-based FL methods.