Faster Secure Multiparty Computation of Adaptive Gradient Descent

Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice Pub Date : 2020-11-09 DOI:10.1145/3411501.3419427

Wen-jie Lu, Yixuan Fang, Zhicong Huang, Cheng Hong, Chaochao Chen, Hunter Qu, Yajin Zhou, K. Ren

引用次数: 9

Abstract

Most of the secure multi-party computation (MPC) machine learning methods can only afford simple gradient descent (sGD 1) optimizers, and are unable to benefit from the recent progress of adaptive GD optimizers (e.g., Adagrad, Adam and their variants), which include square-root and reciprocal operations that are hard to compute in MPC. To mitigate this issue, we introduce InvertSqrt, an efficient MPC protocol for computing 1/√x. Then we implement the Adam adaptive GD optimizer based on InvertSqrt and use it for training on different datasets. The training costs compare favorably to the sGD ones, indicating that adaptive GD optimizers in MPC have become practical.

查看原文

微信好友朋友圈 QQ好友复制链接

本刊更多论文

自适应梯度下降的快速安全多方计算

大多数安全多方计算(MPC)机器学习方法只能提供简单的梯度下降(sgd1)优化器，并且无法从自适应GD优化器(例如Adagrad, Adam及其变体)的最新进展中受益，其中包括在MPC中难以计算的平方根和倒数运算。为了缓解这个问题，我们引入了InvertSqrt，这是一种用于计算1/√x的高效MPC协议。然后我们实现了基于InvertSqrt的Adam自适应GD优化器，并使用它在不同的数据集上进行训练。与sGD的训练成本相比，该方法的训练成本更低，这表明MPC中的自适应GD优化器已经变得实用。

本文章由计算机程序翻译，如有差异，请以英文原文为准。

求助全文

约1分钟内获得全文去求助

来源期刊

Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice

自引率

0.00%

发文量

期刊最新文献

Privacy-Preserving in Defending against Membership Inference Attacks Adversarial Detection on Graph Structured Data Faster Secure Multiparty Computation of Adaptive Gradient Descent SVM Learning for Default Prediction of Credit Card under Differential Privacy Information Leakage by Model Weights on Federated Learning