PATNAS: A Path-Based Training-Free Neural Architecture Search

Abstract

The development of Neural Architecture Search (NAS) is hindered by high costs associated with evaluating network architectures. Recently, several zero-cost proxies have been proposed as a promising method to reduce the evaluation cost of network architectures in NAS. They can quickly estimate the final performance of the network in a few seconds during the initial phase. However, existing zero-cost proxies either ignore the network structure's impact on performance or are limited to specific tasks. To address these issues, we propose a novel zero-cost proxy called Skeleton Path Kernel Trace (SPKT) that leverages the whole network architecture's skeleton path structure information. We then integrate it into an effective Bayesian optimization for NAS framework called PATNAS, and demonstrate its efficacy on different datasets. The results show that our proposed SPKT zero-cost proxy can achieve a high correlation with the final performance of the network across multiple tasks. Furthermore, it can significantly accelerate the search process for finding the best-performing network architectures.