Animation line art colorization based on the optical flow method

Coloring an animation sketch sequence is a challenging task in computer vision since the information contained in line sketches is too sparse, and the colors need to be uniform between continuous frames. Many the existing colorization algorithms can only be applied to one image and can be considered color filling algorithms. Such algorithms only provide a color result that fits within a reasonable range and can not be applied to the coloring of frame sequences. This paper proposes an end-to-end two-stage optical flow colorization network to solve the animation frame sequence colorization problem. The first stage of the network finds the direction of the color pixel flow from the detail change between a given reference frame and the next frame of line artwork and then completes the initial coloring process. The second stage of the network performs color correction and clarifies the output of the first stage. Since our algorithm does not directly colorize the image but finds the path of the color change to colorize it, it ensures a consistent color space for the sequence frames after colorization. We conduct experiments on an animation dataset, and the results show that our algorithm is effective. The code is available at https://github.com/silenye/Colorization.