Heterogeneous Accelerator Design for Multi-DNN Workloads via Heuristic Optimization

The significant advancements of deep neural networks (DNNs) in a wide range of application domains have spawned the need for more specialized, sophisticated solutions in the form of multi-DNN workloads. Heterogeneous DNN accelerators have emerged as an elegant solution to tackle the workloads’ inherent diversity, achieving significant improvements compared to homogeneous solutions. However, utilizing off-the-shelf architectures provides suboptimal adaptability to given workloads, whereas custom design approaches offer limited heterogeneity, and thus reduced gains. In this letter, we combat these shortcomings and propose an exploration-based framework to holistically design heterogeneous accelerators, tailored for multi-DNN workloads. Our framework is workload-agnostic and leverages architectural heterogeneity to its full potential, by integrating low-precision arithmetic and custom structural parameters. We explore the formed design space, targeting to minimize the system’s energy-delay product (EDP) via heuristic techniques. Our proposed accelerators achieve, on average, a significant $5.5\times $ reduction in EDP compared to the state of the art across various multi-DNN workloads.