A Network Behavior-Based Botnet Detection Mechanism Using PSO and K-means

Abstract

In today's world, Botnet has become one of the greatest threats to network security. Network attackers, or Botmasters, use Botnet to launch the Distributed Denial of Service (DDoS) to paralyze large-scale websites or steal confidential data from infected computers. They also employ “phishing” attacks to steal sensitive information (such as users’ accounts and passwords), send bulk email advertising, and/or conduct click fraud. Even though detection technology has been much improved and some solutions to Internet security have been proposed and improved, the threat of Botnet still exists. Most of the past studies dealing with this issue used either packet contents or traffic flow characteristics to identify the invasion of Botnet. However, there still exist many problems in the areas of packet encryption and data privacy, simply because Botnet can easily change the packet contents and flow characteristics to circumvent the Intrusion Detection System (IDS). This study combines Particle Swarm Optimization (PSO) and K-means algorithms to provide a solution to remedy those problems and develop, step by step, a mechanism for Botnet detection. First, three important network behaviors are identified: long active communication behavior (ActBehavior), connection failure behavior (FailBehavior), and network scanning behavior (ScanBehavior). These behaviors are defined according to the relevant prior studies and used to analyze the communication activities among the infected computers. Second, the features of network behaviors are extracted from the flow traces in the network layer and transport layer of the network equipment. Third, PSO and K-means techniques are used to uncover the host members of Botnet in the organizational network. This study mainly utilizes the flow traces of a campus network as an experiment. The experimental findings show that this proposed approach can be employed to detect the suspicious Botnet members earlier than the detection application systems. In addition, this proposed approach is easy to implement and can be further used and extended in the campus dormitory network, home networks, and the mobile 3G network.