ATO-EDGE: Adaptive Task Offloading for Deep Learning in Resource-Constrained Edge Computing Systems

Abstract

On-device deep learning enables mobile devices to perform complex tasks, such as object detection and voice translation, regardless of the network condition. The advanced deep learning model gives an excellent performance, also leads to a heavy burden on resource-limited devices (i.e., mobile devices). To speed up the on-device deep learning. Prior studies focus on developing lightweight network architecture for real-time inference by sacrificing model accuracy. This paper presents ATO-EDGE: adaptive task offloading for deep learning based on edge computing. Considering three optimization goals, energy consumption, accuracy, and latency, ATO-EDGE leverages an offline pre-trained model to select a suitable deep learning model on a specific device to process the given task. We apply our approach to object detection and evaluate it on Jetson TX2, Xilinx ZYNQ 7020, and Raspberry 3B+. The deep learning model candidates contain ten typical object detection models trained on Microsoft COCO 2017 dataset. We obtain, on average, 28.25%, 35.44%, and 0.9 improvements respectively for latency, energy consumption, and mAP (mean average precision) when compared to the SOTA DETR model on the Raspberry Pi.