Low Cost Evolutionary Neural Architecture Search (LENAS) Applied to Traffic Forecasting

Traffic forecasting is an important task for transportation engineering as it helps authorities to plan and control traffic flow, detect congestion, and reduce environmental impact. In recent times, the application of deep learning techniques to handle large and complex datasets has become prevalent. However, these methods necessitate a proficiency in neural architecture engineering, a skill set that many decision makers in traffic management centers may not possess. Neural architecture search (NAS) methods have gained popularity for alleviating the problem of neural architecture engineering by discovering customized neural architectures for various tasks. Their application to traffic prediction has only recently been explored. Performance estimation of neural architectures, a sub-problem of NAS and often the bottleneck in terms of computation time, hinders the adaptation of research to real-world applications. Recently, zero-cost (ZC) proxies have emerged as a cost-effective means of evaluating network architectures without requiring training, circumventing the bottleneck at the expense of accuracy. This work extends previous research on evolutionary NAS (ENAS) by evaluating the utility of ZC proxies for the task of traffic prediction. We answer research questions related to the stability of zero-cost proxies and their correlation with validation losses on real-world datasets. When used in the ENAS framework, we show that ZC proxies can speed up the search process by two orders of magnitude without greatly affecting the accuracy of the prediction model. Traffic forecasting is an important task for transportation engineering as it helps authorities to plan and control traffic flow, detect congestion, and reduce environmental impact. Deep learning techniques have gained traction in handling such complex datasets, but require expertise in neural architecture engineering, often beyond the scope of traffic management decision-makers. Our study aims to address this challenge by using neural architecture search (NAS) methods. These methods, which simplify neural architecture engineering by discovering task-specific neural architectures, are only recently applied to traffic prediction. We specifically focus on the performance estimation of neural architectures, a computationally demanding sub-problem of NAS, that often hinders the real-world application of these methods. Extending prior work on evolutionary NAS (ENAS), our work evaluates the utility of zero-cost (ZC) proxies, recently emerged cost-effective evaluators of network architectures. These proxies operate without necessitating training, thereby circumventing the computational bottleneck, albeit at a slight cost to accuracy. Our findings indicate that, when integrated into the ENAS framework, ZC proxies can accelerate the search process by two orders of magnitude at a small cost of accuracy. These results establish the viability of ZC proxies as a practical solution to accelerate NAS methods while maintaining model accuracy. Our research contributes to the domain by showcasing how ZC proxies can enhance the accessibility and usability of NAS methods for traffic forecasting, despite potential limitations in neural architecture engineering expertise. This novel approach significantly aids in the efficient application of deep learning techniques in real-world traffic management scenarios.