Distributed Deep Learning Approach for Optimal Hyper-Parameter Values

Abstract

With the development of artificial intelligence, there are great changes especially in technology and industry sectors. The fact that deep learning and reinforcement learning studies are popular topics by researchers accelerates this change. In this article, a distributed system is presented to determine the hyper-parameters of the deep learning algorithm used for object detection at the most accurate value. One of the most important factors affecting the accuracy rate in object recognition approaches using deep learning algorithms is the determination of hyper-parameters with correct values. It may be necessary to carry out very long experiments to determine the optimum of these parameters. To solve this problem, a deep learning network used for object detection has been trained by combining the RAY distributed architecture with a deep learning algorithm. The accuracy rate is observed by changing the parameters in each iteration. For object detection, the training of the neural network we created with the CIFAR-10 dataset was carried out using CPU. In addition, thanks to the distributed architecture, each process is trained by 4 different workers. The training results and the properties of the artificial neural network are given in detail in the following sections. Accordingly, we can highlight the main contributions of this article in three points. Firstly; to show that long processes are completed in a short time, thanks to the integration of deep learning algorithms with the distributed system; training the model used to determine the optimal hyper-parameter values and the third is the presentation of the distributed deep learning approach.