Efficient socket-based data transmission method and implementation in deep learning

Abstract

The deep learning algorithm, which has been increasingly applied in the field of petroleum geophysical prospecting, has achieved good results in improving efficiency and accuracy based on test applications. To play a greater role in actual production, these algorithm modules must be integrated into software systems and used more often in actual production projects. Deep learning frameworks, such as TensorFlow and PyTorch, basically take Python as the core architecture, while the application program mainly uses Java, C#, and other programming languages. During integration, the seismic data read by the Java and C# data interfaces must be transferred to the Python main program module. The data exchange methods between Java, C#, and Python include shared memory, shared directory, and so on. However, these methods have the disadvantages of low transmission efficiency and unsuitability for asynchronous networks. Considering the large volume of seismic data and the need for network support for deep learning, this paper proposes a method of transmitting seismic data based on Socket. By maximizing Socket’s cross-network and efficient longdistance transmission, this approach solves the problem of inefficient transmission of underlying data while integrating the deep learning algorithm module into a software system. Furthermore, the actual production application shows that this method effectively solves the shortage of data transmission in shared memory, shared directory, and other modes while simultaneously improving the transmission efficiency of massive seismic data across modules at the bottom of the software.