Efficient federated learning with cross-resource client collaboration

Abstract

Federated learning is a distributed machine learning paradigm. Traditional federated learning performs well on the premise that all clients have the same learning ability or similar learning tasks. However, resource and data heterogeneity are inevitable among clients in real-world scenarios, leading to the situation that existing federated learning mechanisms cannot achieve high accuracy in short response time. In this study, an effective federated learning framework with cross-resource client collaboration, termed CEFL, is proposed to coordinate clients with different capacities to help each other, efficiently and adequately reflecting collective intelligence. Clients are categorized into different clusters based on their computational resources in the hierarchical framework. Resource-rich clusters use their knowledge to assist resource-limited clusters converge rapidly. Once resource-limited clusters have the ability to mentor others, resource-rich clusters learn from the resource-limited clusters in their favor to improve their own effectiveness. A cloud server provides tailored assistance to each cluster with a personalized model through an adaptive multi-similarity metric, in order for each cluster to learn the most useful knowledge. The experiments fully demonstrate that the proposed method not only has superior accuracy with significantly reduced latency but also improves the convergence rate compared to other state-of-the-art federated learning methods in addressing the problem of resource and data heterogeneity.