Selective Updates and Adaptive Masking for Communication-Efficient Federated Learning

Abstract

Federated Learning (FL) is fast becoming one of the most prevalent distributed learning techniques focused on privacy preservation and communication efficiency for large-scale Internet of Things (IoT) deployments. FL is a distributed learning approach to training models on distributed devices. Since local data remains on-device, communication through the network is reduced. However, in large-scale IoT environments or resource constrained networks, typical FL approaches significantly suffer in performance due to longer communication times. In this paper, we propose two methods for further reducing communication volume in resource restricted FL deployments. In our first method, which we term Selective Updates (SU), local models are trained until a dynamic threshold on model performance is surpassed before sending updates to a centralised Parameter Server (PS). This allows for minimal updates being transmitted, thus reducing communication overheads. Our second method, Adaptive Masking (AM), performs parameter masking on both the global and local models prior to sharing. With AM, we select model parameters that have changed the most between training rounds. We extensively evaluate our proposed methods against state-of-the-art communication reduction strategies using two common benchmark datasets, and under different communication constrained settings. Our proposed methods reduce the overall communication volume by over 20%, without affecting the model accuracy.