Demystifying the Membership Inference Attack

Abstract

The Membership Inference Attack (MIA) is the process of determining whether a sample comes from the training dataset (in) of a machine learning model or not (out). This attack makes use of a trained machine learning to expose confidential information about its training data. It is particularly alarming in cases where data is tightly linked to individuals like in the medical, financial and marketing domains. The underlying factors of the success of MIA are not well understood. The current theory explains its success by the difference in the confidence levels for in samples and out samples. In this article, we show that the confidence levels play little to no role in the MIA success in most of the cases. We propose a more general theory that explains previous results and some unexpected observations that have been made in the state-of-the-art. To back up our theory, we run MIA exneriments on MNIST, CIFAR-10 and Fashion-MNIST.