Thermal-to-Color Image Translation for Enhancing Visual Odometry of Thermal Vision

Abstract

A panoptic perception-based generative adversarial network for thermal-to-color image translation is proposed to demonstrate its potential as an image sequence enhancement for monocular visual odometry in blurry and low-resolution thermal vision. The pre-trained panoptic segmentation model is utilized to obtain the panoptic perception (i.e., bounding boxes, categories, and masks) of the image scene to guide the alignment between the object content codes of the original thermal domain and panoptic-level style codes sampled from the target color style space. A feature masking module further refines the style-aligned object representations for sharpening object boundaries to synthesize higher fidelity translated color image sequences. The extensive experimental evaluation shows that our method outperforms other thermal-to-color image translation methods in the image quality of translated color images. We demonstrate that the enhanced image sequences significantly improve the performance of monocular visual odometry compared with dif-ferent competing methods including thermal image sequences.