Enhanced Personality Prediction Using Knowledge Distillation with BERT: A Focus on MBTI

Suman A. Patil, Shivleela Patil, Vijayalaxmi V. Tadkal
{"title":"Enhanced Personality Prediction Using Knowledge Distillation with BERT: A Focus on MBTI","authors":"Suman A. Patil,&nbsp;Shivleela Patil,&nbsp;Vijayalaxmi V. Tadkal","doi":"10.3103/S1060992X2470084X","DOIUrl":null,"url":null,"abstract":"<p>A person’s personality comprises a range of behaviours, attitudes, and emotional patterns that shift throughout time due to ecological and biological influences. Personality prediction from the MBTI dataset poses computational efficiency, memory utilisation, and class imbalance challenges. This study proposes a novel approach leveraging Knowledge Distillation-based BERT to address these challenges. The process involves three stages: pre-processing, feature extraction, and classification. Initially, data is cleaned by removing irrelevant characters and URLs, followed by tokenisation and conversion to lowercase for consistency. The padding ensures uniform input size for DistilBERT, with attention masks aiding focus on relevant tokens. DistilBERT extracts contextual embeddings, enhanced by segment and positional embeddings, capturing semantic meaning via multi-head self-attention. A fully connected layer with GELU activation and batch normalisation mitigates overfitting, followed by a classification layer with Sparsemax activation, addressing the class imbalance. Fine-tuning pre-trained DistilBERT maximises detection accuracy while excluding irrelevant learning objectives. Dynamic masking during inference replaces static masking, and the Radam optimiser optimises hyperparameters for improved convergence. Our approach offers a robust solution that achieves 93% accuracy and 95% F1-score for accurate personality prediction while mitigating computational complexities and class imbalance issues.</p>","PeriodicalId":721,"journal":{"name":"Optical Memory and Neural Networks","volume":"33 4","pages":"455 - 465"},"PeriodicalIF":1.0000,"publicationDate":"2025-02-03","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Optical Memory and Neural Networks","FirstCategoryId":"1085","ListUrlMain":"https://link.springer.com/article/10.3103/S1060992X2470084X","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q4","JCRName":"OPTICS","Score":null,"Total":0}
引用次数: 0

Abstract

A person’s personality comprises a range of behaviours, attitudes, and emotional patterns that shift throughout time due to ecological and biological influences. Personality prediction from the MBTI dataset poses computational efficiency, memory utilisation, and class imbalance challenges. This study proposes a novel approach leveraging Knowledge Distillation-based BERT to address these challenges. The process involves three stages: pre-processing, feature extraction, and classification. Initially, data is cleaned by removing irrelevant characters and URLs, followed by tokenisation and conversion to lowercase for consistency. The padding ensures uniform input size for DistilBERT, with attention masks aiding focus on relevant tokens. DistilBERT extracts contextual embeddings, enhanced by segment and positional embeddings, capturing semantic meaning via multi-head self-attention. A fully connected layer with GELU activation and batch normalisation mitigates overfitting, followed by a classification layer with Sparsemax activation, addressing the class imbalance. Fine-tuning pre-trained DistilBERT maximises detection accuracy while excluding irrelevant learning objectives. Dynamic masking during inference replaces static masking, and the Radam optimiser optimises hyperparameters for improved convergence. Our approach offers a robust solution that achieves 93% accuracy and 95% F1-score for accurate personality prediction while mitigating computational complexities and class imbalance issues.

Abstract Image

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
求助全文
约1分钟内获得全文 去求助
来源期刊
CiteScore
1.50
自引率
11.10%
发文量
25
期刊介绍: The journal covers a wide range of issues in information optics such as optical memory, mechanisms for optical data recording and processing, photosensitive materials, optical, optoelectronic and holographic nanostructures, and many other related topics. Papers on memory systems using holographic and biological structures and concepts of brain operation are also included. The journal pays particular attention to research in the field of neural net systems that may lead to a new generation of computional technologies by endowing them with intelligence.
期刊最新文献
IFDRF: Advancing Anomaly Detection with a Hybrid Machine Learning Model Tracking and Computation of Characteristics of the Movement of People in Groups on Video Using Convolutional Neural Networks Hybrid Network Model for Cardiac Image Segmentation Using MRI Images Magnetic Field-Controlled Phase Transitions in Antiferromagnetic Structures Abnormal Sound Event Detection Method Based on Time-Spectrum Information Fusion
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1