CNN Workloads Characterization and Integrated CPU–GPU DVFS Governors on Embedded Systems

Abstract

Dynamic power management (DPM) techniques on mobile systems are indispensable for deep learning (DL) inference optimization, which is mainly performed on battery-based mobile and/or embedded platforms with constrained resources. To this end, we characterize CNN workloads using object detection applications of YOLOv4/-tiny and YOLOv3/-tiny, and then propose integrated CPU–GPU DVFS governor policies that scale integrated pairs of CPU and GPU frequencies to improve energy–delay product (EDP) with negligible inference execution time degradation. Our results show up to 16.7% EDP improvements with negligible (mostly less than 2%) performance degradation using object detection applications on NVIDIA Jetson TX2.