Differentially private transferrable deep learning with membership-mappings

Abstract

Despite a recent surge of research interest in privacy and transferrable deep learning, optimizing the tradeoff between privacy requirements and performance of machine learning models remains a challenge. This motivates the development of an approach that optimizes both privacy-preservation mechanism and learning of the deep models for achieving a robust performance. This paper considers the problem of semi-supervised transfer and multi-task learning under differential privacy framework. An alternative conception of deep autoencoder, referred to as Conditionally Deep Membership-Mapping Autoencoder (CDMMA), is considered for transferrable deep learning. Under practice-oriented settings, an analytical solution for the learning of CDMMA can be derived by means of variational optimization. The paper proposes a transfer and multi-task learning approach that combines CDMMA with a tailored noise adding mechanism to transfer knowledge from source to target domain in a privacy-preserving manner.