A Study of Developing a Job Recommendation Model Using Transformer Models

Abstract

With the rapid technological advancements brought about by the Fourth Industrial Revolution, the industrial ecosystem is undergoing changes with new professions emerging and existing ones disappearing at an unprecedented rate. This societal shift is leading to an employment market dominated by professionals with experience, with an increasing number of individuals working in roles unrelated to their major. Additionally, the discrepancy between one's major and the job requirements demanded by industries is leading to a rising number of young job seekers abandoning their job search. In light of this, the purpose of this study is to develop job recommendation models based on job seekers' resumes, evaluate the performance of these models, and propose ways to enhance the applicability of the recommended jobs. To achieve this, two experiments were conducted. The first experiment evaluated the performance of a job recommendation model that applied the Transformer architecture, using a training dataset composed of resumes with 52 features and set hyperparameters. The second experiment evaluated the performance of a deep learning model, which applied fine-tuning techniques on a training dataset constructed from 34 features after removing any feature with more than 30% missing values from the original 52 features used in the first experiment. The increase in the epoch value of the Transformer model has significant implications. This research demonstrates that the optimization of training data and the application of fine-tuning in deep learning model design significantly impact the performance improvement of job recommendation models. Given the rate of increase in epoch values observed in this study, it is expected that applying high-quality resume data to the Transformer model for additional model training in the future will further enhance the performance of career-based job recommendation models for both students preparing for employment based on their major and for experienced job seekers.