A Novel Tactile Learning Assistive Tool for the Visually and Hearing Impaired with 3D-CNN and Bidirectional LSTM Leveraging Morse Code Technology.

Abstract

Educating deafblind children is a highly specialized field that requires computer-assisted learning tools to address the challenges of auditory and visual impairments. The objective is to reduce their difficulties in communication with their peers and to empower them to learn independently in a classroom environment. Braille and assistive tools have become profoundly beneficial for deafblind children, serving as an essential means of communication and knowledge acquisition, enabling them to live independently. This study aims to develop an assistive tool that bridges the limitations of conventional tactile methodologies by incorporating the latest artificial intelligence techniques, enabling children to learn with greater ease. The research leverages Morse code technology to facilitate communication with deafblind children. The speaker's lip movements are converted into text using the deep learning techniques of a 3D convolutional neural network and a bidirectional long short-term memory neural network. Experimental evaluations of this text conversion model show a word error rate of 2% and an accuracy rate of 98%. The text is then converted into Morse code and communicated to the deafblind child through a wearable device. The significance of this assistive tool lies in its discreet design, resembling a smartwatch. Adolescents can wear the proposed wearable device confidently without feeling self-conscious or embarrassed.