A Multi-Sensory Blind Guidance System Based on YOLO and ORB-SLAM

Abstract

Blind guidance system has always been a research hotspot for years. Although there are many kinds of blind guidance systems on the market, most of them prompt from the perspective of a single sense of tactile or auditory. The blind guidance method of single sense can be unstable and it does not fully mobilize other general senses of the with vision impairment. This paper designs and implements a multi-sensory blind guidance system that provides tactile and auditory sensations by using the ORB-SLAM and YOLO techniques. Based on the RGB-D camera, the local obstacle avoidance system is realized at the tactile level through the point cloud filtering that feedback the results to the user through vibrating motors. The improved ORB-SLAM can generate a dense navigation map to implement a global obstacle avoidance system through the coordinate transformation. Real-time target detection and the YOLO-based prompt voice system is implemented at the auditory level. The system can detect the specific category and give the location of obstacles as real-time voice messages. The functions mentioned above are integrated and verified as a smart cane. Experimental results show that the position and category of the obstacles in the surrounding environment can be detected accurately in real-time through our system. By combining YOLO and ORB- SLAM, we can provide a piece of useful auxiliary equipment to the community of vision impairment and enable users to move about safely.