Personality Prediction with Natural Language Processing using Questionnaire Responses

Abstract

As the modern IT revolution is booming at a rapid growth speed, organizations and recruiters are finding it increasingly challenging to select the ideal applicant from a large number of applicants with diverse skill sets and personalities. Hence, selecting a candidate with a suitable personality for respective job profiles is a very important and great challenge for the HR department nowadays. Out of various personality prediction methods available out there, Myers-Briggs Type Indicator or MBTI is famous and accurate for our purpose of creating a personality prediction system for selecting candidates based on their personality. This study took into account all sixteen MB-Model coordinates. A comparative study of Random Forest, Logistic Regression, SVM, XGBoost has been done to perform personality prediction, and accuracy and confusion matrix for performance measurement of the models. While using TF-IDF, for the personality categories like Introversion/Extroversion the accuracy is 80.46%, for Sensing/Intuition it is 88.70%, for Thinking/Feeling it is 81.21% and for Perceiving vs Judging it is 72.97% with the Logistic Regression algorithm. Using Count vectorization for tokenizing, the accuracy is 80.97% for Introversion/Extroversion, for Sensing/Intuition it is 88.93%, for Thinking/Feeling it is 77.92% and for Perceiving vs Judging it is 73.48% with XGBoost algorithm, which gave the best performance.