Opto-electrotactile Feedback Enabled Text-line Tracking Control for A Finger-wearable Reading Aid for the Blind

Abstract

In this paper, to achieve the goal of aiding the blind and visually impaired (BVI) to read any text not written in Braille, a custom-built, finger-wearable, and electro-tactile based Braille reading system with its Rapid Optical Character Recognition (R-OCR) method is developed. The R-OCR is capable of processing text information in real time using a miniature fish-eye imaging device mounted at the finger-wearable system. This allows real-time translation of printed text to electro-Braille along with natural movement of user's fingertip as if reading any Braille sign or book. An electro-tactile neuro-stimulation feedback mechanism is further proposed and incorporated with the reading system, which facilitates a new opto-electrotactile-feedback-based text line tracking control approach that enables text line following by user's fingertip during reading. Extensive experiments were designed and conducted to test the ability of blindfolded participants to read through and follow the printed text lines based on this optoelectrotactile-feedback method. The experimental results show that as the outcome of the opto-electrotactile-feedback, the users who involved in the feedback loop were able to maintain their fingertips within a $2mm$ distance of the text while “reading” through a printed text line. Our work is a significant step to aid the BVI users with a portable means to read any printed texts to Braille through the following and the translation, whether in the digital realm or physically, on any surface.