Amplitude-Aligned Personalization and Robust Aggregation for Federated Learning

In practical applications, federated learning (FL) suffers from slow convergence rate and inferior performance resulting from the statistical heterogeneity of distributed data. Personalized FL (pFL) has been proposed to overcome this problem. However, existing pFL approaches mainly focus on measuring differences between entire model dimensions across clients, ignore the layer-wise differences in convolutional neural networks (CNNs), which may lead to inaccurate personalization. Additionally, two potential threats in FL are that malicious clients may attempt to poison the entire federation by tampering with local labels, and the model information uploaded by clients makes them vulnerable to inference attacks. To tackle these issues, 1) we propose a novel pFL approach in which clients minimize local classification errors and align the local and global prototypes for data from the class that is shared with other clients. This method adopts layer-wise collaborative training to achieve more granular personalization and converts local prototypes to the frequency domain to prevent source data leakage; 2) To prevent the FL model from misclassifying certain test samples as expected by poisoners, we design a robust aggregation method to ensure that benign clients who provide trustworthy model predictions for its local data are weighted far more heavily in the aggregation process than malicious clients. Experiments show that our scheme, especially in the data heterogeneity situation, can produce robust performance and more stable convergence while preserving privacy.