Malicious domain detection based on semi-supervised learning and parameter optimization

Abstract

Malicious domains provide malware with covert communication channels which poses a severe threat to cybersecurity. Despite the continuous progress in detecting malicious domains with various machine learning algorithms, maintaining up-to-date various samples with fine-labeled data for training is difficult. To handle these issues and improve the detection accuracy, a novel malicious domain detection method named MDND-SS-PO is proposed that combines semi-supervised learning and parameter optimization. The contributions of the study are as follows. First, the method extracts the statistical features of the IP address, TTL value, the NXDomain record, and the domain name query characteristics to discriminate Domain-Flux and Fast-Flux domain names simultaneously. Second, an improved DBSCAN based on the neighborhood division is designed to cluster labeled data and unlabeled data with low time consumption. Then, based on the clustering hypothesis, unlabeled data is tagged with pseudo-label according to the cluster results, which aims to train a supervised classifier effectively. Finally, Gaussian process regression is used to optimize parameter settings of the algorithm. And the Silhouette index and F1 score are introduced to evaluate the optimization results. Experimental results show that the proposed method achieved a precise detection performance of 0.885 when the ratio of labeled data is 5%.