ChatgaiyyaAlap: A dataset for conversion from Chittagonian dialect to standard Bangla

Abstract

Bangla is one of the most used languages around the world with 240 million speakers. The standard Bangla language is the official language of people from Bangladesh and a few other parts outside Bangladesh, like West Bengal and Tripura. Although, people from different areas of Bangladesh do not use standard Bangla on a day-to-day basis. Instead, dialects of the Bangla language are used. The dialects of the Bangla language are quite diverse which includes the Rajshahi dialect, Sylheti dialect, Old Dhaka dialect, Chittagonian dialect, and many more. Nearly every division of Bangladesh has its unique dialect which adds linguistic diversity to the language. The main difference between the standard Bangla language and Chittagonian dialect is that it does not have any written form and the words vary from the standard Bangla. We built a dataset containing standard Bangla and one of its most used dialects: the Chittagonian dialect. Our dataset, “ChatgaiyyaAlap,” was created by combining 4012 unique sentences in standard Bangla with their Chittagonian translations, which were gathered from dramas, comments, and videos on YouTube and Facebook. In our dataset, we cleaned the data by removing emojis, punctuations, and unessential spaces. The dissimilarity in spelling as well as in sentence structure between standard Bangla and Chittagonian dialects, particularly in negative sentences can be clearly visible through this dataset. Additionally, to maintain the data accuracy, we developed a dictionary containing 1,500 standard Bangla words and their Chittagonian form. The dictionary showcases significant variation in the vocabulary of both languages and resolves ambiguities along with potential biases. To perform a word-to-word translation this dictionary would be useful, while for studying comparative linguistics and developing intelligent systems the ChatgaiyyaAlap dataset might assist the researchers. Moreover, this dataset can be utilized to train language models for linguistic translation.