An Unsupervised Network Anomaly Detection Model and Implementation

Abstract

Anomaly detection for network attacks has always been a very important part of intrusion detection. The current research focus is anomaly detection based on deep learning, which has two main problems. One is the lack of a large amount of labeled data in model training, and the other is difficult to detect unknown network attacks or variant attacks. To solve the above problems, an unsupervised anomaly detection model is constructed in this paper. The automatic encoder is used to learn normal traffic characteristics and detect abnormal traffic. Meanwhile, time correlation features and hierarchical clustering algorithm are used for data preprocessing to reduce time and space complexity, so as to further improve the efficiency of model detection. Due to the serious lack of verification data sets for unsupervised anomaly detection, this paper collects and organizes a large amount of data and designs four types of network attack data, including new attack means, worms, system vulnerabilities and botnets. The experimental results showed that the detection accuracy of worms and system vulnerabilities reached 98%, the detection accuracy of botnets reached 89%, and the attacks of the new OriginLogger software were detected.