Model Recovery in Federated Unlearning With Restricted Server Data Resources

Abstract

Recent model recovery methods in federated unlearning (FUL) either rely on additional communication with the remaining clients or require large amounts of high-quality data from the server for training, overlooking scenarios with limited data resources. Currently, contrastive language-image pretraining (CLIP) has demonstrated remarkable performance across a wide range of tasks, particularly excelling in few-shot learning scenarios. In this article, inspired by CLIP, we explore the scenario of few-shot knowledge distillation and propose CLIP-guided few-shot knowledge distillation (CGKD) for model recovery in FUL. CGKD mainly consists of three components: 1) the unlearning module constructs the unlearning model by erasing all historical contributions of the target client, and this model is treated as the student model; 2) fine-tuning the pretrained CLIP model using few-shot data from the server side to obtain a more robust teacher model (CLIP $^{\mathbf {*}}$ ); and 3) model recovery is achieved through knowledge distillation, leveraging the rich visual and semantic knowledge of CLIP $^{\mathbf {*}}$ to enhance the student model’s understanding of image semantic context, thereby improving the performance of the unlearning model. Extensive experimental results demonstrate that CGKD outperforms the compared FUL method in recovery performance across four standard datasets, validating the effectiveness of our approach.