Contention-aware Performance Modeling for Heterogeneous Edge and Cloud Systems

Abstract

Diversely Heterogeneous System-on-Chips (DH-SoC) are increasingly popular computing platforms in many fields, such as autonomous driving and AR/VR applications, due to their ability to effectively balance performance and energy efficiency. Having multiple target accelerators for multiple concurrent workloads requires a careful runtime analysis of scheduling. In this study, we examine a scenario that mandates several concerns to be carefully addressed: 1) exploring the mapping of various workloads to heterogeneous accelerators to optimize the system for better performance, 2) analyzing data from the physical world in runtime to minimize the response time of the system 3) accurately estimating the resource contention by workloads during runtime since there will be con- current operations running under the same die, and 4) deferring the operation to the cloud for computationally more demanding operations such as continuous learning or real-time rendering, de- pending on the complexity of the computation. We demonstrate our analysis and approach on a VR project as a case study by using NVIDIA Xavier NX Edge DH-SoC and a server equipped with NVIDIA GeForce RTX 3080 GPU and AMD EPYC 7402 CPU.