Hyper-parameter Tuning of Federated Learning Based on Particle Swarm Optimization

Abstract

The learning task of federated learning (FL) is solved by a federation of a center server and individual clients. Contrary to traditional deep learning models, federated learning consists of two parts, the global model and individual models. However, since the process is carried out by two parts of equal importance, it is ideal to deal with both parts simultaneously. To achieve superior performance, federated learning requires carefully selected hyper-parameters, which has more hyper-parameters than those of traditional deep learning models. To solve this tuning problem, we propose a method using particle swarm optimization (PSO) algorithm to tune the hyperparameters of federated learning. PSO algorithm is a gradient-free, stochastic optimization method which is better than grid search method when it comes to large search space. It helps locate the optimal combination of multiple parameters. In this article, we focus on applying PSO method on tuning the hyper-parameters of FL models, and prove that it is an efficient way to acquire satisfactory results. Experiments on MNIST dataset with convolution neural networks have proved the superiority of the proposed method.