{"title":"Communication-efficient and privacy-preserving large-scale federated learning counteracting heterogeneity","authors":"Xingcai Zhou, Guang Yang","doi":"10.1016/j.ins.2024.120167","DOIUrl":null,"url":null,"abstract":"<div><p>Federated learning is a commonly distributed framework for large-scale learning, where a model is learned over massively distributed remote devices without sharing information on devices. It has at least three key challenges: heterogeneity in federated networks, privacy and communication costs. In this paper, we propose three federated learning algorithms to handle these issues gradually. First, we introduce a FedSfDane algorithm (<strong>DANE</strong> with <strong>S</strong>hrinkage <strong>f</strong>actor for <strong>Fed</strong>erated learning), which improves the inexact approximation of the full gradient, captures statistical heterogeneity and restrains systems heterogeneity across the devices. For avoiding possible privacy leakage in federated learning, a <strong>P</strong>rivacy-preserving FedSfDane algorithm (PFedSfDane) is proposed, which is resistant to adversary attacks. Further, we give a novel <strong>C</strong>ommunication-efficient PFedSfDane (CPFedSfDane) algorithm for large-scale federated networks, which effectively handles the above three challenges. We give convergence guarantees for the three algorithms to convex and non-convex learning problems. Numerical experiments illustrate our algorithms outperform FedDANE, FedAvg and FedProx algorithms, especially for highly heterogeneous federated networks. CPFedSfDane improves the prediction accuracy of the state-of-the-art FedDANE algorithm by about 15.0% on sent140 dataset, and has high privacy protection and communication efficiency.</p></div>","PeriodicalId":51063,"journal":{"name":"Information Sciences","volume":null,"pages":null},"PeriodicalIF":8.1000,"publicationDate":"2024-01-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Information Sciences","FirstCategoryId":"94","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S002002552400080X","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"0","JCRName":"COMPUTER SCIENCE, INFORMATION SYSTEMS","Score":null,"Total":0}
引用次数: 0
Abstract
Federated learning is a commonly distributed framework for large-scale learning, where a model is learned over massively distributed remote devices without sharing information on devices. It has at least three key challenges: heterogeneity in federated networks, privacy and communication costs. In this paper, we propose three federated learning algorithms to handle these issues gradually. First, we introduce a FedSfDane algorithm (DANE with Shrinkage factor for Federated learning), which improves the inexact approximation of the full gradient, captures statistical heterogeneity and restrains systems heterogeneity across the devices. For avoiding possible privacy leakage in federated learning, a Privacy-preserving FedSfDane algorithm (PFedSfDane) is proposed, which is resistant to adversary attacks. Further, we give a novel Communication-efficient PFedSfDane (CPFedSfDane) algorithm for large-scale federated networks, which effectively handles the above three challenges. We give convergence guarantees for the three algorithms to convex and non-convex learning problems. Numerical experiments illustrate our algorithms outperform FedDANE, FedAvg and FedProx algorithms, especially for highly heterogeneous federated networks. CPFedSfDane improves the prediction accuracy of the state-of-the-art FedDANE algorithm by about 15.0% on sent140 dataset, and has high privacy protection and communication efficiency.
期刊介绍:
Informatics and Computer Science Intelligent Systems Applications is an esteemed international journal that focuses on publishing original and creative research findings in the field of information sciences. We also feature a limited number of timely tutorial and surveying contributions.
Our journal aims to cater to a diverse audience, including researchers, developers, managers, strategic planners, graduate students, and anyone interested in staying up-to-date with cutting-edge research in information science, knowledge engineering, and intelligent systems. While readers are expected to share a common interest in information science, they come from varying backgrounds such as engineering, mathematics, statistics, physics, computer science, cell biology, molecular biology, management science, cognitive science, neurobiology, behavioral sciences, and biochemistry.