Multi-task ADAS system on FPGA

Abstract

Advanced Driver-Assistance Systems (ADAS) can help drivers in the driving process and increase the driving safety by automatically detecting objects, doing basic classification, implementing safeguards, etc. ADAS integrate multiple subsystems including object detection, scene segmentation, lane detection, and so on. Most algorithms are now designed for one specific task, while such separate approaches will be inefficient in ADAS which consists of many modules. In this paper, we establish a multi-task learning framework for lane detection, semantic segmentation, 2D object detection, and orientation prediction on FPGA. The performance on FPGA is optimized by software and hardware co-design. The system deployed on Xilinx zu9 board achieves 55 FPS, which meets real-time processing requirement.