ASMAFL: Adaptive Staleness-Aware Momentum Asynchronous Federated Learning in Edge Computing

Abstract

Compared with synchronous federated learning (FL), asynchronous FL (AFL) has attracted more and more attention in edge computing (EC) fields because of its strong adaptability to heterogeneous application scenarios. However, the non-independent and identically distributed (Non-IID) data across devices and the staleness-aware estimation of unreliable wireless connections and limited edge resources make it much more difficult to achieve better AFL-related applications. To handle this problem, we propose an Adaptive Staleness-aware Momentum Accelerated AFL (ASMAFL) algorithm to reduce the resources consumption of heterogeneous wireless communication EC (WCEC) scenarios, as well as decrease the negative impact of Non-IID data for model training. Specifically, we first introduce the staleness-aware parameter and a unified momentum gradient descent (GD) framework to reformulate AFL. Then, we establish global convergence properties of AFL, derive an upper bound on AFL convergence rate, and find that the bound is related to the staleness-aware parameter and Non-IIDness. Next, we formulate the bound into a minimization problem of resource consumption under given model accuracy, and the corresponding staleness-aware parameter of devices will be recomputed after each asynchronous aggregation to eliminate the differences of local models’ contribution to global model aggregation. Finally, extensive experiments are carried out to validate the superiority of ASMAFL in model accuracy, convergence rate, resources consumption, Non-IID issue, etc.