Sentiment Analysis For Arabic Low Resource Data Using BERT-CNN

Abstract

Users share opinions and discussions on the internet through social media platforms. Nowadays, a significant number of internet users speak the Arabic language. They tend to express their opinions using different dialects. Therefore, understanding people's opinions and emotions become an urgent matter. The Arabic sentiment analysis is challenging because of linguistic complexity, data availability, and data quality, and it has multiple dialects. Therefore, research for low resources sentiment analysis became necessary. This study proposes a Bidirectional Encoder Representations from Transformers (BERT) that uses Convolutional Neural Network (CNN) as a classification head for Arabic low data resources for sentiment analysis. The classification head includes the CNN layer, drop-out layer, and a Relu activation function. The proposed approach experimented on three datasets collected from Twitter containing different dialects. The last four BERT layers were fined-tuned and while other layers were frozen. The suggested model outperforms current state-of-the-art models' accuracy with 50% fewer batch size, fewer training layers, and ∼20% fewer epochs.