Understanding Crowds' Migration on the Web

Abstract

Consider a network where nodes are websites and the weight of a link that connects two nodes corresponds to the average number of users that visits both of the two websites over longer timescales. Such user-driven Web network is not only invaluable for understanding how crowds' interests collectively spread on the Web, but also useful for applications such as advertising or search. In this paper, we manage to construct such a network by 'putting together' pieces of information publicly available from the popular analytics websites. Our contributions are threefold. First, we design a crawler and a normalization methodology that enable us to construct a user-driven Web network based on limited publicly-available information, and validate the high accuracy of our approach. Second, we evaluate the unique properties of our network, and demonstrate that it exhibits small-world, seed-free, and scale-free phenomena. Finally, we build an application, website selector, on top of the user-driven network. The core concept utilized in the website selector is that by exploiting the knowledge that a number of websites share a number of common users, an advertiser might prefer displaying his ads only on a subset of these websites to optimize the budget allocation, and in turn increase the visibility of his ads on other websites. Our websites elector system is tailored for ad commissioners and it could be easily embedded in their ad selection algorithms.