DPSGD Strategies for Cross-Silo Federated Learning

Abstract

As federated learning (FL) grows and new techniques are created to improve its efficiency and robustness, differential privacy (DP) proved to be a good ally for protecting users’ information. The differentially private version of stochastic gradient descent (DPSGD) is one of the most promising methods for enforcing privacy in machine learning algorithms. The noise added in DPSGD plays an important role in the convergence and performance of a model but also in the resulting privacy guarantee and must thus be chosen carefully. This paper reviews the effects of either selecting fixed or adaptive noise when training federated models under the cross-silo setting. We highlight their strengths and weaknesses and propose a hybrid approach, getting the best of both worlds.