Endoscopic Image Colorization Using Convolutional Neural Network

Abstract

Colorization of grayscale images is crucial for clinical image-based diagnosis. However, it is an ill-posed problem that requires a comprehensive understanding of image content. The present study proposes a novel convolutional neural network (CNN) for a fully automatic colorization process by first employing the pre-trained residual network to extract high-level image features and then introducing the CNN to analyze the complex nonlinear relationship between the image features and chrominance values. Luminance and the learned chrominance values are then combined to recover the color of the image, and the proposed color-perceptual loss function is used to calculate the recovered and real color image loss. Based on the experiments conducted, the proposed method was proven to be highly effective and robust in restoring endoscopic images to their true colors. The average values of the feature similarity index incorporating chromatic information (FSIMc) and the quaternion structural similarity (QSSIM) for the experimental endoscopic image datasets reached 0.9961 and 0.9739, respectively.