Implementation of Edge-cloud Cooperative CNN Inference on an IoT Platform

Abstract

Since the Internet of Things (IoT) has become more widely used in various industrial situations, Artificial Intelligence (AI) programs, particularly Convolutional Neural Network (CNN) applications, are projected to be implemented on edge devices to meet high-accuracy and huge industry computing needs. Offloading computing-intensive workloads to the cloud is a promising solution for compact energy-constrained edge devices, but it tends to incur significant costs in total execution latency. For flexible and fine-grained offloading, this paper aims to design and implement an edge-cloud cooperative CNN inference framework on an IoT platform by targeting TensorFlow Lite. We have confirmed the implementation's feasibility and accuracy through the verification of implementing LeNet, AlexNet, and VGGNet. Intending to perform high-performance edge-cloud AI executions on the presented IoT platform, we evaluate the performance overhead (total execution latency) of the provided implementation and identify the current bottlenecks of the target platform for enhancing it.