Split-Knit Convolution: Enabling Dense Evaluation of Transpose and Dilated Convolutions on GPUs

Abstract

Transpose convolutions occur in several image-based neural network applications, especially those involving segmentation or image generation. Unlike regular (forward) convolutions, they result in data access and computation patterns that are less regular, and generally have poorer performance when implemented in software. We present split-knit convolution (SKConv) – a technique to replace transpose convolutions with multiple regular convolutions followed by interleaving. We show how existing software frameworks for GPU implementation of deep neural networks can be adapted to realize this computation, and compare against the standard techniques used by such frameworks.