An autonomous elevator button recognition system based on convolutional neural networks

The ability to operate an elevator is significant for service robots to move freely inside a building and the elevator button recognition is placed as one of the most critical functions of this process. However, the variety of button styles, the different light conditions and the blurred images caused by the camera motion make this task difficult. To tackle this obstacle to achieve the robust real-time performance, a button recognition system is proposed based on the convolutional neural networks. In consideration of the diverse button shapes, a contour extraction algorithm and the noise filtering are specifically designed to avoid the exhaustive search and reduce the consumed time. Then the fine-tuned CNN model is trained on our established elevator button dataset to achieve a more reliable recognition performance comparing to the template matching methods. Besides, the arrangement pattern of buttons is utilized to deduce the missing buttons and correct mistakes. To verify our algorithm, we run our algorithm on a dataset of 5 distinct elevators. Our algorithm succeeds in localizing and recognizing 98% of the buttons in known elevators and 87.6% in unknown elevators and has an average speed of 3 frames per second.