FedPnP: Personalized graph-structured federated learning

Abstract

In Personalized Federated Learning (PFL), current methods often fail to consider the fine-grained relationships between clients and their local datasets, hindering effective information exchange. Here, we propose “FedPnP”, a novel method that harnesses the inherent graph-based connections among clients. Clients linked by a graph tend to yield similar model responses to comparable input data. In the proposed FedPnP we present the graph-based optimization as an inverse problem. We then solve this optimization by employing a Half-Quadratic-Splitting technique (HQS) to divide it into two subproblems. The first ensures local model performance on respective datasets, acting as a data fidelity term, while the second promotes the smoothness of model weights on the graph. Notably, we present a structural proximal term in the first subproblem and demonstrate the integration of any graph denoiser in the second subproblem as a plug & play solution. Experiments on CIFAR10, CIFAR100, FashionMNIST, and SVHN demonstrate FedPnP’s superiority over 10 state-of-the-art algorithms, with accuracy improvements ranging from 0.2% to 3%. Notably, FedPnP excels in handling highly heterogeneous data, a critical challenge in real-world PFL scenarios. Additional evaluations show that FedPnP performs consistently well across various denoisers, with the Heat filter delivering the best results. This bridge between PFL algorithms and inverse problems opens up the potential for cross-pollination of solutions, yielding superior algorithms for PFL tasks. The GitHub code is available at https://github.com/arashrasti96/FedPnP.