Modular deep learning-based network intrusion detection architecture for real-world cyber-attack simulation

In an increasingly digitalized world, cybersecurity has emerged as a critical component of safeguarding sensitive information and infrastructure from malicious threats. The threat actors are often in line or even one step ahead of the defense, causing the increasing reliance of security teams on artificial intelligence while trying to detect zero-day attacks. However, most of the cybersecurity solutions based on artificial intelligence that can be found in the literature are trained and tested on reference datasets that are at least five or more years old, which gives a vague insight into their security performances. Moreover, they often tend to be designed as isolated, self-focused components. The aim of this paper is to design and implement a modular network intrusion detection architecture capable of simulating cyberattacks based on real-world scenarios while evaluating its defense capabilities. The architecture is designed as a full pipeline from real-time network data collection and transformation to threat-information presentation and visualization, with a pre-trained artificial intelligence module at its core. Well-known components like CICFlowMeter, Prometheus, and Grafana are used and modified to fit our data preparation and core modules to form the proposed architecture for real-world network traffic security monitoring. For the sake of cyberattack simulation, the proposed architecture is situated within a virtual environment, surrounded by the Kali Linux-based penetration simulation agent on one side and a vulnerable agent on the other. The intrusion detection artificial intelligence module is trained on the CICIDS-2017 dataset, and it is demonstrated using the proposed architecture that, despite being trained on an outdated dataset, the trained module is still effective in detecting sophisticated modern attacks. Two case studies are given to illustrate how modular architectures and virtual environments can be valuable tools to assess the security properties of artificial intelligence-based solutions through simulation in real-world scenarios.