Improving distributed learning-based vulnerability detection via multi-modal prompt tuning

Abstract

Software vulnerabilities pose significant threats to the integrity and reliability of complex systems, making their detection critical. In recent years, a growing body of research has explored deep learning-based approaches for identifying vulnerabilities, which have shown promising results. However, many of these methods ignore privacy and security issues. We utilize distributed learning techniques that enable local models to interact without data sharing. By aggregating these locally trained models, we can update the global model while maintaining data privacy and security. Additionally, existing methods rely on a single source of code semantic information. However, leveraging multiple modalities can capture diverse code representations and features. Specifically, graph-based representations and source code provide structural and syntactic-semantic information that complements traditional code analysis. In this study, we propose a novel function-level vulnerability detection approach MIVDL. It integrates both structured and unstructured features of source code. Then, it further combines the code token sequence with the Code Property Graph (CPG) for enhanced detection accuracy. This hybrid representation leverages the strengths of different modalities to provide a comprehensive understanding of code semantics. Furthermore, our approach employs a pre-trained model applied to distinct parts of each modality before being integrated into a single hybrid representation. This allows a unified analysis framework to utilize each modality’s unique features and strengths. Additionally, distributed learning facilitates collaborative learning and knowledge-sharing among participating entities. We evaluate MIVDL on three datasets (Devign, Reveal, and Big-Vul), and the results indicate that MIVDL outperformed eight state-of-the-art baselines by 3.04$\sim$70.73% in terms of F1-score. Therefore, combining multi-modal prompt tuning and distributed learning can improve performance in vulnerability detection.