Intelligent detection of vulnerable functions in software through neural embedding-based code analysis

Abstract

Software vulnerability is a fundamental problem in cybersecurity, which poses severe threats to the secure operation of devices and systems. In this paper, we propose a new vulnerability detection framework of employing advanced neural embedding. For example, CodeBERT is a large-scale pre-trained embedding model for natural language and programming language. It achieves state-of-the-art performance on various natural language processing and code analysis tasks, demonstrating improved generalization ability compared with conventional models. The proposed framework encapsulates CodeBERT as a code representation generator and combines it with transfer learning to conduct cross-project vulnerability detection. Considering the problem of lacking code embedding models on C source code, we extract the knowledge from C source code to fine-tune the pre-trained embedding model, so as to better facilitate the detection of function-level vulnerabilities in C open-source projects. To address the severe data imbalance issue in real-world scenarios, we introduce code argumentation idea and use a large number of synthetic vulnerability data to further improve the robustness of the detection method. Experimental results show that the proposed vulnerability detection framework achieves better performance than existing methods.