Synergistic Distributed CNN Model for Protein Classification With a Collaborative BSP Synchronization Based on LSTM Prediction

Abstract

Recently, distributed deep learning has been introduced as the new highly computational solution that could handle huge amounts of data and reduce training time. Especially when handling high-dimensional and complicated data, is very challenging, such as dealing with Genomics which is the most demanding in terms of data acquisition, storage, distribution, and analysis. However, Distributed deep learning has issues that need to be resolved. Focusing on the synchronization paradigm, BSP (Bulk Synchronous Parallel) is the most used model. Even so, it is demanding in terms of time due to an exigent problem called the straggler, where all the workers need to wait for the slowest worker to synchronize. Therefore, in this article, we propose a collaborative BSP (Collab-BSP) that aims to solve this issue by adopting LSTM for execution time prediction and implementing it with the Apache Spark environment. We proved the efficiency of our approach in reducing the waiting time and iteration time by 50% and 30%, respectively. Also, our approach demonstrated promising results while training a distributed CNN for protein classification with 98.82% accuracy and proved its capability to enhance distributed deep learning training.