Vulnerable Road Users Detection and Tracking using YOLOv4 and Deep SORT

Abstract

Over the years, The detection and tracking of Vulnerable Road Users (VRUs) have become one of the most critical features of self-driving car components. Because of its processing efficiency and better detection algorithms, tracking-by-detection appears to be the best paradigm. In this paper, a detection-based tracking approach is presented for Multiple VRU Tracking of video from an inside-vehicle camera in real-time. YOLOv4 scans every frame to detect VRUs first, then Simple Online and Realtime Tracking with a Deep Association Metric (Deep SORT) algorithm, which is customized for multiple VRU tracking, is applied. The results of our experiments on both the Joint Attention in Autonomous Driving (JAAD) and Multiple Object Tracking (MOT) datasets exhibit competitive performance.