An autoencoder-based fast online clustering algorithm for evolving data stream

Abstract

In the era of Big Data, more and more IoT devices are generating huge amounts of high-dimensional, real-time and dynamic data streams. As a result, there is a growing interest in how to cluster this data effectively and efficiently. Although a number of popular two-stage data stream clustering algorithms have been proposed, these algorithms still have some problems that are difficult to solve in the face of real-world data streams: poor handling of high-dimensional data streams and difficulty in effective dimensionality reduction; a slow clustering process that makes it difficult to meet real-time requirements; and too many manually defined parameters that make it difficult to cope with evolving data streams. This paper proposes an autoencoder-based fast online clustering algorithm for evolving data stream(AFOCEDS). The algorithm uses a stacked denoising autoencoder to reduce the dimensionality of the data, a multi-threaded approach to improve response speed, and a mechanism to automatically update parameters to cope with evolving data streams. The experiments on several realistic data streams show that AFOCEDS outperforms other algorithms in terms of effectiveness and speed.