On dynamic data clustering and visualization using swarm intelligence

Abstract

Clustering and visualizing high-dimensional sparse data simultaneously is a very attractive goal, yet it is also a challenging problem. Our previous studies using a special type of swarms, known as flocks of agents, provided some promising approaches to this challenging problem on several limited size UCI machine learning data sets and Web usage sessions (from web access logs) [1], [2]. However, dynamic domains, such as practically any data generated on the Web, may require frequent costly updates of the clusters (and the visualization), whenever new data records are added to the dataset. The new coming data may be due to new user activity on a website (clickstreams) or a search engine (queries), or new Web pages in the case of document clustering, etc. Additionally, data records may result in a change of clustering in time. Therefore, clusters may need to be updated, thus leading to the need to mine dynamic clusters. This paper summarizes our initial studies in designing a simultaneous clustering and visualization algorithm and proposes the Dynamic-FClust Algorithm, which is based on flocks of agents as a biological metaphor. This algorithm falls within the swarm-based clustering family, which is unique compared to other approaches, because its model is an ongoing swarm of agents that socially interact with each other, and is therefore inherently dynamic.