A Hypergraph-Based Machine Learning Ensemble Network Intrusion Detection System

Abstract

Network intrusion detection systems (NIDSs) to detect malicious attacks continue to meet challenges. NIDS are often developed offline while they face auto-generated port scan infiltration attempts, resulting in a significant time lag from adversarial adaption to NIDS response. To address these challenges, we use hypergraphs (HGs) focused on Internet protocol (IP) addresses and destination ports to capture evolving patterns of port scan attacks. The derived set of HG-based metrics are then used to train an ensemble machine learning (ML)-based NIDS that allows for real-time adaption in monitoring and detecting port scanning activities, other types of attacks, and adversarial intrusions at high accuracy, precision and recall performances. This ML adapting NIDS was developed through the combination of 1) intrusion examples; 2) NIDS update rules; 3) attack threshold choices to trigger NIDS retraining requests; and 4) a production environment with no prior knowledge of the nature of network traffic. 40 scenarios were auto-generated to evaluate the ML ensemble NIDS comprising three tree-based models. The resulting ML ensemble NIDS was extended and evaluated with the CIC-IDS2017 dataset. Results show that under the model settings of an Update-ALL-NIDS rule (specifically retrain and update all the three models upon the same NIDS retraining request) the proposed ML ensemble NIDS evolved intelligently and produced the best results with nearly 100% detection performance throughout the simulation.