C++OpenCL4TVM: Support C++OpenCL Kernel for TVM NN Operators

Abstract

In an era of artificial intelligence (AI), OpenCL serves as one of the AI frameworks’ back-ends, notably, the tensor virtual machine (TVM), which focuses on the inference side of neural networks. After optimizing a computational graph, TVM traverses the internal representations, Tensor-level IR (TIR), of each neural network (NN) operator generating OpenCL kernels for each one of them. In this work, we make TVM generate C++ for OpenCL, compile it to SPIR-V binary, and consume it with clCreateProgramWithIL inside TVM after we transform it by adding C[2]++ for_each and providing unseq as its argument. We also bumped into an llvm-spirv issue along the way. Finally, we found a workaround and proceeded to runnable TVM-generated C++ for OpenCL kernels.