A novel and highly efficient botnet detection algorithm based on network traffic analysis of smart systems

In smart systems, attackers can use botnets to launch different cyber attack activities against the Internet of Things. The traditional methods of detecting botnets commonly used machine learning algorithms, and it is difficult to detect and control botnets in a network because of unbalanced traffic data. In this article, we present a novel and highly efficient botnet detection method based on an autoencoder neural network in cooperation with decision trees on a given network. The deep flow inspection method and statistical analysis are first applied as a feature selection technique to select relevant features, which are used to characterize the communication-related behavior between network nodes. Then, the autoencoder neural network for feature selection is used to improve the efficiency of model construction. Finally, Tomek-Recursion Borderline Synthetic Minority Oversampling Technique generates additional minority samples to achieve class balance, and an improved gradient boosting decision tree algorithm is used to train and establish an abnormal traffic detection model to improve the detection of unbalanced botnet data. The results of experiments on the ISCX-botnet traffic dataset show that the proposed method achieved better botnet detection performance with 99.10% recall, 99.20% accuracy, 99.1% F1 score, and 99.0% area under the curve.