Deep Color Constancy Using Spatio-Temporal Correlation of High-Speed Video

Abstract

After the invention of electric bulbs, most of lights surrounding our worlds are powered by alternative current (AC). This intensity variation can be captured with a high-speed camera, and we can utilize the intensity difference between consecutive video frames for various vision tasks. For color constancy, conventional methods usually focus on exploiting only the spatial feature. To overcome the limitations of conventional methods, a couple of methods to utilize AC flickering have been proposed. The previous work employed temporal correlation between high-speed video frames. To further enhance the previous work, we propose a deep spatio-temporal color constancy method using spatial and temporal correlations. To extract temporal features for illuminant estimation, we calculate the temporal correlation between feature maps where global features as well as local are learned. By learning global features through spatio-temporal correlation, the proposed method can estimate illumination more accurately, and is particularly robust to noisy practical environments. The experimental results demonstrate that the performance of the proposed method is superior to that of existing methods.