DMRP: Privacy-Preserving Deep Learning Model with Dynamic Masking and Random Permutation

Abstract

Large AI models exhibit significant efficiency and precision in addressing complex problems. Despite their considerable advantages in various domains, these models encounter numerous challenges, notably high training costs. Currently, the training of distributed large AI models offers a solution to mitigate these elevated costs. However, distributed large AI models remain susceptible to data reconstruction attacks. A malicious server could leverage the intermediate results uploaded by clients to reconstruct the original data within the framework of distributed large AI models. This study first examines the underlying principles of data reconstruction attacks and proposes a privacy protection scheme. Our approach begins by obfuscating the mapping relationship between embeddings and the original data to ensure privacy protection. Specifically, during the upload of embedding data by clients to the server, genuine embeddings are concealed to prevent unauthorized access by malicious servers. Building on this concept, we introduce DMRP, a defensive mechanism featuring Dynamic Masking and Random Permutation, designed to mitigate data reconstruction attacks while maintaining the accuracy of the primary task. Our experiments, conducted across three models and four datasets, demonstrate the effectiveness of DMRP in countering data reconstruction attacks within distributed large-scale AI models.