Single-Photon Cameras Image Reconstruction Using Vision Transformer

Abstract

Single-photon camera is a novel camera type that utilizes image sensor with photon-counting capability. Recently, the potential of such sensors to achieve high spatial resolutions (e.g., 10^9pixels/chip) and frame rates (e.g., 10^6frames/sec) is well-established. However, there is a significant difference in the output data between single-photon cameras and traditional CMOS sensor cameras. Therefore, conventional image reconstruction algorithms cannot be utilized. Vison Transformer has impressive performance on image processing tasks, and this paper will design a plug-and-play algorithm for reconstructing images from single-photon cameras and integrate image processing algorithms based on deep neural networks like ViT. The results demonstrate our methods can achieve improvement both in PSNR and SSIM.