A New Network Intrusion Detection based on Semi-supervised Dimensionality Reduction and Tri-LightGBM

Abstract

With the development of technology and threat forms, network intrusion detection has become a challenging task. The intrusion detection algorithm based on supervised learning requires a lot of manpower and material resources to obtain a large amount of labeled data. Besides, the accuracy of unsupervised learning can not meet the requirements of intrusion detection systems. We propose a semi-supervised network intrusion detection method in this paper. Information Gain is employed to filter redundancy features. Then, we combine labeled samples with unlabeled samples and adopt Principal Component Analysis (PCA) to convert multiple features into comprehensive features. Finally, an Tri-Training strategy is adopted to integrate the basic LightGBM classifier, and make full use of unlabeled data to generate pseudo labels, thereby optimizing the basic LightGBM classifier. To verify the effectiveness of the proposed approach, a large number of experiments are performed on the UNSW-NB15 dataset. The experimental results fully show that the method is superior in improving detection efficiency and reducing label dependence, and has a lower false alarm rate and a higher detection rate.