Network Intrusion Detection: A comparative study using state-of-the-art machine learning methods

Abstract

Cyber threats are not only increasing with the years, but are also becoming harder to recognize and evolving with time so that they can easily bypass normal antivirus. There have been numerous cyber crimes that have attacked confidentiality and privacy of data. To ensure network security, an effective intrusion detection system is required. Several ensemble methods like XG-Boost and LGBM have been developed in the past 4-5 years. These have not been exploited in the previous researches on anomaly detection. This study makes use of these novel Gradient Boosting Decision Tree algorithms. XG-Boost and LGBM have proved to be the most productive techniques for several supervised and unsupervised learning tasks. This research studies several machine learning and deep learning classifiers and compare their performances. To predict the probability of occurrence of 21 different classes of attacks on a network the NSL KDD dataset has been used. We studied three different categories of models-Linear Models including Logistic Regression and Stochastic Gradient Descent (SGD) classifier; Gradient Boosting Decision Tree ensembles including Light GBM (LGBM) and XG-Boost; and a Deep Neural Network (DNN) classifier and also trained a stacked model consisting of all these models as base learners. This study compares the performances of all the models for Network Intrusion Detection and useful conclusions are drawn. The simulation results show that ensemble methods are more effective for detecting network intrusion.