Heterogeneous Computing and Applications in Deep Learning: A Survey

With the rapid development of deep learning, a variety of neural network models emerge in endlessly, which leads to a huge demand for computing resources. For the intensive numerical computation of neural networks, various computing devices represented by GPUs are favored by researchers. Heterogeneous computing is a kind of technology that can integrate a variety of computing devices with different architectures, and it will be further developed. Therefore, this paper reviews research on some key technologies of heterogeneous computing, including the architecture of heterogeneous computing, the programming language of heterogeneous computing, and the scheduling algorithm for heterogeneous systems. Then, we focus on the research of heterogeneous computing in deep learning, including the parallel technology of neural networks and optimization technology based on heterogeneous systems. Finally, the present research situation is discussed and analyzed, and the future research direction is prospected, aiming to provide some basis for related research.