Conservative vs. Optimistic Parallelization of Stateful Network Intrusion Detection

Abstract

This paper presents and experimentally analyzes the performance of three parallelization strategies for the popular open-source Snort network intrusion detection system (NIDS). The parallelizations include 2 conservative variants and 1 optimistic scheme. The conservative strategy parallelizes inspection at the level of TCP/IP flows, as any potential inter-packet dependences are confined to a single flow. The flows are partitioned among threads, and each flow is processed in-order at one thread. A second variation reassigns flows between threads to improve load balance but still requires that only one thread process a given flow at a time. The flow-concurrent scheme provides good performance for 3 of the 5 network packet traces studied, reaching as high as 4.1 speedup and 3.1 Gbps inspection rate on a commodity 8-core server. Dynamic reassignment does not improve performance scalability because it introduces locking overheads that offset any potential benefits of load balancing. Neither conservative version can achieve good performance, however, without enough concurrent networkflows. For this case, this paper presents an optimistic parallelization that exploits the observation that not all packets from a flow are actually connected by dependences. This system allows a single flow to be simultaneously processed by multiple threads, stalling if an actual dependence is found. The optimistic version has additional overheads that reduce speedup by 25% for traces with flow concurrency, but its benefits allow one additional trace to see substantial speedup (2.4 on five cores).