Toward end-to-end object detection and tracking on the edge

This work presents an architecture solution for enabling real-time vision processing on the edge next to video cameras. We demonstrate the benefits of proposed solution by constructing and real-time processing of complete vision applications for real-time object detection and tracking. It combines six challenging vision kernels including image smoothing, Mixture of Gaussians (MoG) background subtraction, morphology (dilation and erosion), component labeling, histogram checking and Kalman filter. We prototype the proposed solution on a Xilinx Zynq-based platform processing 1080p frames at 30Hz. It executes 40GOPs at only 1.7Watts of on-chip power, far beyond the processing capabilities of state-of-the-art vision processing platforms.