Performance Evaluation of YOLOv3 and YOLOv4 Detectors on Elevator Button Dataset for Mobile Robot

Abstract

The performance evaluation of an AI network model is the important part for building an effective solution before its deployment in real-world on the robot. In our study, we have implemented YOLOv3-tiny and YOLOv4-tiny darknet based frameworks for performance evaluation of the elevator button recognition task and tested both variants on image and video datasets. The objective of our study is two-fold: First, to overcome the limitation of elevator buttons dataset by creating new dataset and increasing its quantity without compromising the quality; Second, to provide a comparative analysis through experimental results and the performance evaluation of both detectors using four machine learning metrics. The purpose of our work is to assist the researchers and developers in decision making of suitable detector selection for deployment in the elevator robot towards button recognition application. The results show that YOLOv4-tiny outperforms YOLOv3-tiny with an overall accuracy of 98.60% compared to 97.91% at 0.5 IoU.