Evaluating Machine Unlearning: Applications, Approaches, and Accuracy

Abstract

Machine learning (ML) enables computers to learn from experience by identifying patterns and trends. Despite ML's advancements in extracting valuable data, there are instances necessitating the removal or deletion of certain data, as ML models can inadvertently memorize training data. In many cases, ML models may memorize sensitive or personal data, raising concerns about data privacy and security. Machine unlearning (MU) techniques offer a solution to these concerns by selectively removing sensitive data from trained models without significantly compromising their performance. Similarly, we can analyze and evaluate whether MU can successfully achieve the “right to be forgotten.” In this paper, we investigate various MU approaches regarding their accuracy and potential applications. Experiments have shown that the data-driven approach emerged as the most efficient method in terms of both time and accuracy, achieving a high level of precision with a minimal number of training epochs. When fine-tuning, the full test error rises somewhat to 14.57% from the baseline model's 14.28%. One approach shows a high forget error of 99.90% with a full test error of 20.68%, while retraining yields a 100% forget error and a test error of 21.37%. While error-minimizing noise preserves performance, the SCRUB technique results in a 21.08% test error and an 81.05% forget error, in contrast to the degradation brought on by error-maximizing noise. On the other hand, the agnostic approach displayed sluggishness and generated less accurate results compared to the data-driven approach. Furthermore, the choice of approach may depend on the unique requirements of the task and the available training resources.