Depression Intensity Identification using Transformer Ensemble Technique for the Resource-constrained Bengali Language

Abstract

Depression is an ordinary mental health-related disorder that hampers people’s daily activities, and sometimes, it destroys an individual’s life. It is one of the major social issues at present. Since depressed people use various social networking sites for sharing their thoughts and feelings, many scholars have tried to identify depression texts in highly resourced languages like English; however, only a small quantity of papers are detected in the resource-constrained Bengali language. This paper focuses on developing a depression intensity detection system from Bengali text data. In this regard, this study experiments on a 2,596 sample-sized dataset with four levels of depression by utilizing five state-of-the-art transformer models, including multilingual Bidirectional Encoder Representations from Transformers, DistilmBERT, XLM-RoBERTa, Bangla-BERT-Base, and BanglaBERT, and suggests a new ensemble method called MaxOfAvgProb. This method goes beyond the performance of the previous work on the same dataset, scoring 63.47% F1-score and 62.90% accuracy. To increase the reliability of the proposed method, we utilize this approach on another available dataset with 4,897 entries. In this case, our recommended method also surpasses the performance of the existing work on the same dataset, with accuracy at 86.45% and F1-score at 86.35%. Identifying the intensity of depression, depressed people may get proper counseling or treatment from their respected guardians or psychologists according to the victims’ level of depression.