Evolutionary Neural Architecture Search with Semi-supervised Accuracy Predictor

Abstract

Neural architecture search (NAS) has low efficiency in evaluating a large number of candidate architectures. As an efficient evaluation method, accuracy predictor-based NAS algorithms have become popular because the performance (accuracy) can be evaluated without training the candidate architectures. However, accuracy predictors still need some evaluated architectures that are difficult to train for achieving promising performance. In order to break this bottleneck, we investigate a semi-supervised accuracy predictor-based evolutionary NAS method (MSNAS) which requires only a small number of evaluated neural architectures. The accuracy predictor obtains high prediction performance by extracting the evaluated architectures, strong regressors and truncation mechanism. To find truly high-accuracy candidate architectures more easily, the multi-objective optimization method is presented to trade-off the prediction accuracy and confidence of candidate architectures. The MSNAS variants from different strong regressors are employed to validate the competitive performance of the MSNAS on NAS-Bench 201.