Binary Lightweight Neural Networks for Arbitrary Scale Super-Resolution of Remote Sensing Images

Abstract

Super-resolution (SR) of remote sensing images (RSIs) has been improved significantly with the development of deep learning. However, better performances usually come from complex network architectures and require a substantial number of parameters. Moreover, many methods can only deal with SR of single and fixed-scale factors. As such, we propose a binary lightweight SR (BLiSR) method to decrease the computation and storage burden and increase the practicality of SR, where we employ a binary neural network (BNN) as the backbone and leverage a binary continuous up-sampling module (BCUM) to achieve arbitrary scale RSI SR. Specifically, we introduce an adaptive binary convolution (ABConv) as the basic unit of BLiSR, which can adaptively adjust the learnable parameters to fit the distribution of full-precision weights and activations. Then, a scalable hyperbolic tangent function is presented to approximate the Sign function in backpropagation and increase the learning capability of BNN. Furthermore, we design a lightweight SR network that considers the full-precision information flow of BNN. The network comprises several basic binary units and a multilayer group fusion block (MGFB), which can extract and fuse the multilevel information from LR images, respectively. Finally, BCUM can predict the pixel values of HR images based on the frequency implicit representation network (IRN) and reconstruct the LR images at arbitrary scales. Extensive experiments on four RSI datasets demonstrate that the proposed BLiSR is superior to several lightweight state-of-the-art (SOTA) methods on both fixed and arbitrary scale SR settings, with a better balance of complexity and performance.