Incremental learning with self-organizing maps

Abstract

We present a novel use for self-organizing maps (SOMs) as an essential building block for incremental learning algorithms. SOMs are very well suited for this purpose because they are inherently online learning algorithms, because their weight updates are localized around the best-matching unit, which inherently protects them against catastrophic forgetting, and last but not least because they have fixed model complexity limiting execution time and memory requirements for processing streaming data. However, in order to perform incremental learning which is usually supervised in nature, SOMs need to be complemented by a readout layer as well as a self-referential control mechanism for prototype updates in order to be protected against negative consequences of concept drift. We present the PROPRE architecture which implements these functions, thus realizing incremental learning with SOMs in very high-dimensional data domains, and show its capacity for incremental learning on several known and new classification problems. In particular, we discuss the required control of SOM parameters in detail and validate our choices by experimental results.