Energy-Efficient, Delay-Constrained Edge Computing of a Network of DNNs

Abstract

This paper presents a novel approach for executing the inference of a network of pre-trained deep neural networks (DNNs) on commercial-off-the-shelf devices that are deployed at the edge. The problem is to partition the computation of the DNNs between an energy-constrained and performance-limited edge device $\boldsymbol{\mathcal{E}}$, and an energy-unconstrained, higher performance device $\boldsymbol{\mathcal{C}}$, referred to as the cloudlet, with the objective of minimizing the energy consumption of $\boldsymbol{\mathcal{E}}$ subject to a deadline constraint. The proposed partitioning algorithm takes into account the performance profiles of executing DNNs on the devices, the power consumption profiles, and the variability in the delay of the wireless channel. The algorithm is demonstrated on a platform that consists of an NVIDIA Jetson Nano as the edge device $\boldsymbol{\mathcal{E}}$ and a Dell workstation with a Titan Xp GPU as the cloudlet. Experimental results show significant improvements both in terms of energy consumption of $\boldsymbol{\mathcal{E}}$ and processing delay of the application. Additionally, it is shown how the energy-optimal solution is changed when the deadline constraint is altered. Moreover, the overhead of decision-making for our proposed method is significantly lower than the state-of-the-art Integer Linear Programming (ILP) solutions.