Deep unsupervised clustering by information maximization on Gaussian mixture autoencoders

Abstract

Clustering has been extensively studied in data mining and machine learning, with numerous applications across domains. In this paper, we propose the Gaussian Mixture Autoencoder (GMAE), a deep clustering method that integrates a probabilistic Autoencoder (AE) with a Gaussian Mixture Model (GMM). GMAE trains the GMM to model the latent representation distribution of the AE and further regularizes the aggregated posterior distribution by minimizing a KL divergence-based loss. To prevent degenerate solutions and enhance clustering performance, a negative mutual information loss is introduced in the model. Additionally, a package of strategies, including an initialization method, an adjusted loss function and an alternating iterative method, is designed to optimize the loss function effectively. Beyond clustering, GMAE can generate diverse, realistic samples for any target cluster, as it trains a decoder with reconstruction loss and adopts the GMM to regularize the latent representation distribution. Experiments on five cross-domain benchmarks demonstrate superior performance over state-of-the-art clustering methods.