Relative density estimation using Self-Organizing Maps

Abstract

Organizations need knowledge of change, such as changes in customer purchasing behaviour, to adapt business strategies in response to changing circumstances. To understand what has changed, analysts have to be able to relate new knowledge acquired from a newer dataset to that acquired from an earlier dataset. This paper presents a method to detect changes in clustering structure over time. Discovering clustering changes can also be applied in other contexts, such as fraud detection and customer attrition analysis. The key contribution of this paper is the enhancement of the measurement of relative density using SOM. This measurement is used in the visualization method called Relative Density Self-Organizing Map (ReDSOM) to compare clustering structures from two snapshot datasets. This visualization provide means for analysts to visually identify and analyze various changes in the clustering structure, such as emerging clusters, disappearing clusters, splitting clusters, and merging clusters. These contributions have been evaluated using synthetic datasets, as well as real-life datasets from the World Bank. Experiments showed that the new measure is more sensitive in detecting changes in density.