Sample analysis and multi-label classification for malicious sample datasets

Abstract

Network attacks pose serious threats to cybersecurity. Researchers provide well-known malicious sample datasets for evaluating methods to detect these attacks. However, we discover that these datasets exhibit a multi-label phenomenon, where a sample has multiple labels. Multi-label problems are ubiquitous, such as in malware detection, where different engines could assign different labels to the same unknown software. But multi-label phenomenon in computer network datasets is different from the traditional multi-label problem. These datasets, which are by default single-labeled, annotated, published, and utilized to evaluate various single-label detection methods. Researchers ignore the possibility that the samples within the datasets may be multi-labeled. Therefore, it is inappropriate to directly utilize these data for evaluating single-label detection methods.

In this paper, we focus on well-known malicious traffic and malware datasets with a comprehensive study, including sample analysis and multi-label classification: (1) We perform comprehensive statistics on 15 datasets, quantify the proportion of multi-label samples and the number of categories affected in them, and analyze the intrinsic connections between attacks. (2) We employ multiple classical multi-label algorithms to classify the multi-label samples in 9 datasets, and the experimental results show that they are superior to the single-label state-of-the-art (SOTA) method, and can improve accuracy and F1 by 39.6% and 57.69% on average.

We conclude that the multi-label phenomenon is ubiquitous in malicious traffic and malware datasets, and it should be considered in network attack detection.