Cracking the Smart ClickBot

Nowadays, almost every task involving Web traversing and information retrieval recurs to Web robots. Web robots are software programs that automatically traverse the Web's hypertext structure. They proliferate rapidly aside with the growth of the Web and are extremely valuable and important means not only for the large search engines, but also for many specialized services such as investment portals, competitive intelligence tools, etc. While many web robots serve useful purposes, recently, there have been cases linked to fraudulent activities committed by these Web robots. Click fraud, which is the act of generating illegitimate clicks, is one of them. This paper details the architecture and functionality of the Smart ClickBot, a sophisticated software bot that is designed to commit click fraud. It was first detected and reported by NetMosaics Inc. in March, 2010, a real time click fraud detection and prevention solution provider. We discuss the machine learning algorithms used, to identify all clicks exhibiting Smart ClickBot like patterns. We constructed a Bayesian classifier that automatically classifies server log data as being Smart ClickBot or not. We also introduce a Benchmark data set for Smart ClickBot. We disclose the results of our investigation of this bot to educate the security research community and provide information regarding the novelties of the attack.