VulDefend: A Novel Technique based on Pattern-exploiting Training for Detecting Software Vulnerabilities Using Language Models

Abstract

The detection of vulnerabilities in source code is a critical task in software assurance. In this work, we propose a semi-supervised learning approach that leverages pattern-exploiting training and cloze-style questions. Our approach involves training a language model on the SARD and Devign datasets of code snippets with vulnerabilities, where the input is generated by masking parts of the code and asking the model to predict the masked tokens. Experimental results demonstrate that our approach can effectively detect vulnerabilities in source code, while leveraging the pattern information learned from the code snippets. This work highlights the feasibility of using pattern-exploiting training and cloze-style questions for improved performance in the detection of vulnerabilities in source code.