Rangegram: A novel payload based anomaly detection technique against web traffic

Abstract

Application specific intrusion detection methods are used to detect network intrusions targeted at applications. Normally such detection methods require payload or packet content analysis. One of the prominent method of payload modeling and analysis is sequence or ngram modeling. Normally ngrams generated from a packet are compared with a database of ngrams seen during training phase. Depending on the number of ngrams found or not found in the packet it is labeled either as normal or anomalous. Previous methods use either presence or absence of ngram in training dataset or use frequency of its occurrence in the entire training dataset. This approach results into many false positives and false negatives. In this paper we propose a novel payload analysis technique for the detection of Zero day attacks against web traffic. We consider the minimum and maximum occurrence frequency of a particular ngram from a packet in training dataset and find deviations from this range to detect anomalies. Experiments on a large dataset has shown good detection rate with low false positives.