No-Clean-Reference Image Super-Resolution: Application to Electron Microscopy

Abstract

The inability to acquire clean high-resolution (HR) electron microscopy (EM) images over a large brain tissue volume hampers many neuroscience studies. To address this challenge, we propose a deep-learning-based image super-resolution (SR) approach to computationally reconstruct a clean HR 3D-EM image with a large field of view (FoV) from noisy low-resolution (LR) acquisition. Our contributions are I) investigation of training with no-clean references; II) introduction of a novel network architecture, named EMSR, for enhancing the resolution of LR EM images while reducing inherent noise. The EMSR leverages distinctive features in brain EM images–repetitive textural and geometrical patterns amidst less informative backgrounds– via multiscale edge-attention and self-attention mechanisms to emphasize edge features over the background; and, III) comparison of different training strategies including using acquired LR and HR image pairs, i.e., real pairs with no-clean references contaminated with real corruptions, pairs of synthetic LR and acquired HR, as well as acquired LR and denoised HR pairs. Experiments with nine brain datasets showed that training with real pairs can produce high-quality super-resolved results, demonstrating the feasibility of training with nonclean references. Additionally, comparable results were observed, both visually and numerically, when employing denoised and noisy references for training. Moreover, utilizing the network trained with synthetically generated LR images from HR counterparts proved effective in yielding satisfactory SR results, even in certain cases, outperforming training with real pairs. The proposed SR network was compared quantitatively and qualitatively with several established SR techniques, demonstrating either the superiority or competitiveness of the proposed method in recovering fine details while mitigating noise.