A Framework for Automatic Generation of FAQs from Email Repositories

Abstract

In many organizations, enquiry emails from customers remain unanswered due to lack of patience and availability of a respondent. Generating FAQs from email repositories with lot of enquiry emails will be beneficial. However, manual generation of FAQs by experts is a time consuming and strenous job. Hence automatic generation of FAQs is a necessity. Automatic generation of FAQs require effective categorization of emails which is challenging since the emails are written by different people with heterogenous cognition levels. In this paper, we propose a framework using Non-negative Matrix Factorization (NMF) and k-means that groups emails into clusters which can be used for FAQ generation. The proposed framework determines not only the broad topic under which the emails have to be tagged but also categorizes the emails into clusters with similar sub contents. The number of clusters was determined by the elbow method whereas the number of topics was fixed by calculating the percentage of relevant topics. The average Silhouette coefficient score of the resulting clusters was found to be 0.52 indicating reasonably good clusters. Also, the Silhouette coefficient score of the proposed method increased by 36.82 % compared to k-means.