Distributed Learning of Pure Non-IID data using Latent Codes

Abstract

There has been a huge increase in the amount of data being generated as a result of the proliferation of high-tech, data-generating devices made possible by recent developments in mobile technology. This has rekindled interest in creating smart applications that can make use of the possibilities of this data and provide insightful results. Concerns about bandwidth, privacy, and latency arise when this data from many devices is aggregated in one location to create more precise predictions. This research presents a novel distributed learning approach, wherein a Variational Auto Encoder is trained locally on each client and then used to derive a sample set of points centrally. The server then develops a unified global model, and sends its training parameters to all users. Pure non-i.i.d. distributions, in which each client only sees data labelled with a single value, are the primary focus of our study. According to our findings, communication amongst the server and the clients takes significantly less time than it does in federated and centralised learning setups. We further demonstrate that, whenever the data is spread in a pure non-iid fashion, our methodology achieves higher accuracy than the federated learning strategy by more than 4%. We also showed that, in comparison to centralised and federated learning systems, our suggested method requires less network bandwidth.