HCT: Hybrid CNN-Transformer Networks for Super-Resolution

Abstract

Recently, several computer vision tasks have begun to adopt transformer-based approaches with promising results. Using a completely transformer-based architecture in image recovery achieves better performance than the existing CNN approach, but the existing vision transformers lack the scalability for high-resolution images, which means that transformers are underutilized in image restoration tasks. We propose a hybrid architecture (HCT) that uses both CNN and transformer to improve image restoration. HCT consists of transformer and CNN branches. By fully integrating the two branches, we strengthen the network's ability of parameter sharing and local information aggregation, and also increase the network's ability to integrate global information, and finally achieve the purpose of improving the image recovery effect. Our proposed transformer branch uses a spatial fusion adaptive attention model that blends local and global attention improving image restoration while reducing computing costs. Extensive experiments show that HCT achieves competitive results in super-resolution tasks.