An Effective Multi-Swarm Algorithm for Optimizing Hyperparameters of DNN

Abstract

Different hyperparameter settings for a deep neural network (DNN) algorithm will come up with different prediction results. One of the most important things is thus in selecting a set of suitable hyperparameters for a DNN so as to increase its accuracy. This can be regarded as a hyperparameter optimization problem for DNN or DNN-based algorithms. Compared with manual, grid search, or random search for parameter settings, metaheuristic algorithms are able to find better hyperparameters for DNNs. To improve the accuracy of a prediction model based on DNN, an improved version of multi-swarm particle swarm optimization (MSPSO) is presented in this paper. Moreover, data provided by Taipei Rapid Transit Corporation will be used to evaluate the performance of the proposed algorithm in predicting the number of passengers for the Taipei metro station. The simulation results show that the proposed algorithm can be used to find better hyperparameters for DNN. This means that the proposed algorithm can provide a more accurate result than other machine learning algorithms, DNN, and PSO with DNN in terms of the prediction accuracy.