How Private is Machine Learning?

Abstract

A machine learning model is private if it doesn't reveal (too much) about its training data. This three-part talk examines to what extent current networks are private. Standard models are not private. We develop an attack that extracts rare training examples (for example, individual people's names, phone numbers, or addresses) out of GPT-2, a language model trained on gigabytes of text from the Internet [2]. As a result there is a clear need for training models with privacy-preserving techniques. We show that InstaHide, a recent candidate, is not private. We develop a complete break of this scheme and can again recover verbatim inputs [1]. Fortunately, there exists provably-correct "differentiallyprivate" training that guarantees no adversary could ever succeed at the above attacks. We develop techniques to that allow us to empirically evaluate the privacy offered by such schemes, and find they may be more private than can be proven formally [3].