A two-stage federated learning method for personalization via selective collaboration

Abstract

As an emerging distributed learning method, Federated learning has received much attention recently. Traditional federated learning aims to train a global model on a decentralized dataset, but in the case of uneven data distribution, a single global model may not be well adapted to each client, and even the local training performance of some clients may be superior to the global model. Under this background, clustering resemblance clients into the same group is a common approach. However, there is still some heterogeneity of clients within the same group, and general clustering methods usually assume that clients belong to a specific class only, but in real-world scenarios, it is difficult to accurately categorize clients into one class due to the complexity of data distribution. To solve these problems, we propose a two-stage federated learning method for personalization via selective collaboration (FedSC). Different from previous clustering methods, we focus on how to independently exclude other clients with significant distributional differences for each client and break the restriction that clients can only belong to one category. We tend to select collaborators for each client who are more conducive to achieving local mission goals and build a collaborative group for them independently, and every client engages in a federated learning process only with group members to avoid negative knowledge transfer. Furthermore, FedSC performs finer-grained processing within each group, using an adaptive hierarchical fusion strategy of group and local models instead of the traditional approach’s scheme of directly overriding local models. Extensive experiments show that our proposed method considerably increases model performance under different heterogeneity scenarios.