UrduHope: Analysis of hope and hopelessness in Urdu texts

Hope is a crucial aspect of human psychology that has received considerable attention due to its role in facing challenges in human life. However, current research predominantly focuses on hope as positive anticipation, overlooking its counterpart, hopelessness. This paper addresses this gap by presenting an expanded framework for analyzing hope speech in social media, incorporating hope and hopelessness. Drawing on insights from psychology and Natural Language Processing (NLP), we argue that a comprehensive understanding of human emotions necessitates considering both constructs. We introduce the concept of hopelessness as a distinct category in hope speech analysis and develop a novel dataset for Urdu, an underrepresented language in NLP research. We proposed a semi-supervised annotation procedure by utilizing Large Language Models (LLMs) along with human annotators to annotate the dataset and explored various learning approaches for hope speech detection, including traditional machine learning models, neural networks, and state-of-the-art transformers. The findings demonstrate the effectiveness of different learning approaches in capturing the nuances of hope speech in Urdu social media discourse. The hope speech detection task was modeled in two subtasks: a binary classification of Urdu tweets to Hope and Not Hope classes and then a multiclass classification of Urdu tweets into Generalized, Realistic, and Unrealistic Hopes, along with Hopelessness, and Not Hope (Neutral) categories. The best results for binary classification were obtained with Logistic Regression (LR) with an averaged macro F1 score of 0.7593, and for the multiclass classification experiments, transformers outperformed other experiments with an averaged macro F1 score of 0.4801.