LipBengal: Pioneering Bengali lip-reading dataset for pronunciation mapping through lip gestures

Abstract

The LipBengal dataset represents a significant advancement in Bengali lip-reading and visual speech recognition research, poised to drive future applications and technological progress. Despite Bengali's global status as the seventh most spoken language with approximately 265 million speakers, linguistically rich and widely spoken languages like Bengali have been largely overlooked by the research community. LipBengal fills this gap by offering a pioneering dataset tailored for Bengali lip-reading, comprising visual data from 150 speakers across 54 classes, encompassing Bengali phonemes, alphabets, and symbols. Captured under diverse and uncontrolled conditions, LipBengal stands as the most extensive Bengali lip-reading dataset to date, designed to facilitate robust benchmarking and validation of novel deep learning architectures. Detailed annotations extend from phoneme- level classifications to full sentence constructions, providing a granular and comprehensive dataset. The primary potential of LipBengal lies in its thorough coverage of Bengali phonemes, capturing diverse lip movements linked to distinct sounds. This rich dataset holds promise for training accurate lip-reading models, with implications for improved accessibility, enhanced speech recognition, silent speech interfaces, and linguistic research. The dataset's diversity in speaker backgrounds enhances its utility, ensuring broader representation of Bengali pronunciation patterns. Meticulous annotation and curation further bolster its quality and reliability, making LipBengal a valuable asset for researchers and developers in the field.