Network Security Systems Log Analysis for Trends and Insights: A Case Study

Network perimeter security appliances like firewalls, intrusion detection systems mediate communications and log details pertaining to various events. Logs generated by these systems are used to identify security compromises, vulnerable systems, mis-configurations, etc and serve as a valuable asset for a network administrator. In this paper, we report on a study conducted using logs generated by production level security appliances deployed in our university network. In particular, we process the logs generated by firewall, intrusion detection/prevention system and domain name system service to identify trends and gain insights. We process 71 million network connection records which includes 95.7 thousand alerts generated by an open source intrusion detection system collected over a period of 31 days and derive statistics to understand end host level behavioral trends. In our analysis we compare hosts which are known to be infected with malware or running Peer-to-Peer applications and remaining using a set of relevant parameters and identify clearly differentiated behavioral trends.