DCTM: Dual Contrastive Topic Model for identifiable topic extraction

Abstract

The recent advanced Contrastive Neural Topic Model (CNTM) was proposed to tackle topic collapse through document-level contrastive learning. However, limited by its usage of the Logistic-Normal prior in topic space and document level contrastive learning, it is less capable of disentangling semantically similar topics. To address the limitation, we propose a novel Dual Contrastive Topic Model (DCTM) that utilizes the Dirichlet prior to capture interpretable patterns. Besides, it incorporates dual (document-level and topic-level) contrastive learning on the topic distribution matrix which helps generate discriminative topic representations and mine identifiable topics. Our proposed DCTM outperforms the state-of-the-art neural topic models in terms of topic coherence and diversity, which is verified by extensive experimentation on three publicly available text corpora. In detail, the proposed DCTM surpasses baselines on almost all the used topic coherence metrics ($C_P$, $C_A$, NPMI for 20Newsgroups, $C_P$, $C_A$, NPMI and UCI for Grolier and DBPedia), and it also obtains higher topic diversity with 1 datasets respectively. Moreover, when performing text clustering, DCTM also achieves significant improvements, with observed increases of more than 1% (20Newsgroups) and 6% (DBPedia) in accuracy.