Neural architecture search for resource constrained hardware devices: A survey

Abstract

With the emergence of powerful and low-energy Internet of Things devices, deep learning computing is increasingly applied to resource-constrained edge devices. However, the mismatch between hardware devices with low computing capacity and the increasing complexity of Deep Neural Network models, as well as the growing real-time requirements, bring challenges to the design and deployment of deep learning models. For example, autonomous driving technologies rely on real-time object detection of the environment, which cannot tolerate the extra latency of sending data to the cloud, processing and then sending the results back to edge devices. Many studies aim to find innovative ways to reduce the size of deep learning models, the number of Floating-point Operations per Second, and the time overhead of inference. Neural Architecture Search (NAS) makes it possible to automatically generate efficient neural network models. The authors summarise the existing NAS methods on resource-constrained devices and categorise them according to single-objective or multi-objective optimisation. We review the search space, the search algorithm and the constraints of NAS on hardware devices. We also explore the challenges and open problems of hardware NAS.