Personalized Federated Learning with Gradient Similarity

Abstract

In the conventional federated learning, the local models of multiple clients are trained independently by their privacy data, and the center server generates the shared global model by aggregating local models. However, the global model often fails to adapt to each client due to statistical heterogeneities, such as non-IID data. To address the problem, we propose the Subclass Personalized Federated Learning (SPFL) algorithm for non-IID data. In SPFL, the server uses the Softmax Normalized Gradient Similarity (SNGS) to weight the relationship between clients, and sends the personalized global model to each client. The stage strategy of ResNet is also applied to improve the performance of our algorithm. The experimental results show that the SPFL algorithm used on non-IID data outperforms the vanilla FedAvg, Per-FedAvg, FedUpdate, and pFedMe algorithms, improving the accuracy by 1.81∼18.46% on four datasets (CIFAR10, CIFAR100, MNIST, EMNIST), while still maintaining the state-of-the-art performance on IID data.