Extracting DNN Architectures via Runtime Profiling on Mobile GPUs

Abstract

Deep Neural Networks (DNNs) have become invaluable intellectual property for AI providers due to advancements fueled by a decade of research and development. However, recent studies have demonstrated the effectiveness of model extraction attacks, which threaten this value by stealing DNN models. These attacks can lead to misuse of personal data, safety risks in critical systems, and the spread of misinformation. This paper explores model extraction attacks on DNN models deployed on mobile devices, using runtime profiles as a side-channel. Since mobile devices are resource constrained, DNN deployments require optimization efforts to reduce latency. The main hurdle in extracting DNN architectures in this scenario is that optimization techniques, such as operator-level and graph-level fusion, can obfuscate the association between runtime profile operators and their corresponding DNN layers, posing challenges for adversaries to accurately predict the computation performed. To overcome this, we propose a novel method analyzing GPU call profiles to identify the original DNN architecture. Our approach achieves full accuracy in extracting DNN architectures from a predefined set, even when layer information is obscured. For unseen architectures, a layer-by-layer hyperparameter extraction method guided by sub-layer patterns is introduced, also achieving high accuracy. This research achieves two firsts: 1) targeting mobile GPUs for DNN architecture extraction and 2) successfully extracting architectures from optimized models with fused layers.