Estimating Similarity between Visual and Long Wave Infrared patches using Siamese CNN

Abstract

Image matching is the process of identifying correspondences between same scene images that differ due to different acquisition parameters such as illumination, viewpoint, or noise. Image patch matching involves computing similarity between the patches based on content invariant to various photometric or geometric variations. Our objective is to design a convolution neural network that computes similarity between visual and infrared image patches of same scene. Similarities of images are measured from the feature maps that are extracted from raw patches. A model is developed that maps the patch to low-dimensional feature vector and similarity is calculated using a fully connected layer which outputs the distance between patches. Threshold is applied on the similarity resulting ‘1’ for similar patches and ‘0’ for dis-similar patches. Siamese CNN architecture based on transfer learning with regression is built with convolution trained and tested for patch similarity. Network model is trained with illumination varying patches of Hpatches dataset and are evaluated with a dataset of corresponding visual and long wave infrared images.