Shize Gao;Guoqing Wang;Baorong Xie;Xin Wei;Jue Wang;Wenchao Liu
{"title":"用于遥感图像连续超分辨率的标频双调制方法","authors":"Shize Gao;Guoqing Wang;Baorong Xie;Xin Wei;Jue Wang;Wenchao Liu","doi":"10.1109/JSTARS.2024.3483991","DOIUrl":null,"url":null,"abstract":"In recent years, the development of continuous-scale super-resolution (SR) methods in the field of remote sensing (RS) has garnered significant attention. These innovative methods are capable of delivering arbitrary-scale image SR through a single unified network. However, the majority of these methods employ the same feature extractor for different SR scales, which constrains the enhancement of network performance. Furthermore, the utilization of a multilayer perceptron for image reconstruction results in the loss of a substantial quantity of high-frequency information, which is of particular significance in the context of RS images. This, in turn, gives rise to the generation of blurred SR results. In order to address the aforementioned issues, the scale-frequency dual-modulation network (SFMNet) is proposed as a means of achieving RS image continuous SR. First, scale modulation feature fusion can modulate different levels of feature fusion according to different scale factors, thereby fully integrating the scale information into the feature extraction process of the network. Subsequently, frequency modulation reconstruction can modulate the frequency-domain information at the root of the image reconstruction process, thereby enhancing the ability of the network to learn high-frequency information. The experimental results demonstrate that the proposed SFMNet outperforms existing RS image continuous SR methods in terms of quantitative indices and visual quality.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19682-19697"},"PeriodicalIF":4.7000,"publicationDate":"2024-10-21","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10723306","citationCount":"0","resultStr":"{\"title\":\"Scale-Frequency Dual-Modulation Method for Remote Sensing Image Continuous Super-Resolution\",\"authors\":\"Shize Gao;Guoqing Wang;Baorong Xie;Xin Wei;Jue Wang;Wenchao Liu\",\"doi\":\"10.1109/JSTARS.2024.3483991\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"In recent years, the development of continuous-scale super-resolution (SR) methods in the field of remote sensing (RS) has garnered significant attention. These innovative methods are capable of delivering arbitrary-scale image SR through a single unified network. However, the majority of these methods employ the same feature extractor for different SR scales, which constrains the enhancement of network performance. Furthermore, the utilization of a multilayer perceptron for image reconstruction results in the loss of a substantial quantity of high-frequency information, which is of particular significance in the context of RS images. This, in turn, gives rise to the generation of blurred SR results. In order to address the aforementioned issues, the scale-frequency dual-modulation network (SFMNet) is proposed as a means of achieving RS image continuous SR. First, scale modulation feature fusion can modulate different levels of feature fusion according to different scale factors, thereby fully integrating the scale information into the feature extraction process of the network. Subsequently, frequency modulation reconstruction can modulate the frequency-domain information at the root of the image reconstruction process, thereby enhancing the ability of the network to learn high-frequency information. The experimental results demonstrate that the proposed SFMNet outperforms existing RS image continuous SR methods in terms of quantitative indices and visual quality.\",\"PeriodicalId\":13116,\"journal\":{\"name\":\"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing\",\"volume\":\"17 \",\"pages\":\"19682-19697\"},\"PeriodicalIF\":4.7000,\"publicationDate\":\"2024-10-21\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10723306\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing\",\"FirstCategoryId\":\"5\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10723306/\",\"RegionNum\":2,\"RegionCategory\":\"地球科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"ENGINEERING, ELECTRICAL & ELECTRONIC\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","FirstCategoryId":"5","ListUrlMain":"https://ieeexplore.ieee.org/document/10723306/","RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"ENGINEERING, ELECTRICAL & ELECTRONIC","Score":null,"Total":0}
引用次数: 0
摘要
近年来,遥感(RS)领域连续尺度超分辨率(SR)方法的发展备受关注。这些创新方法能够通过单一的统一网络提供任意尺度的图像超分辨率。然而,这些方法大多采用相同的特征提取器来处理不同尺度的 SR,这限制了网络性能的提升。此外,使用多层感知器进行图像重建会导致大量高频信息丢失,这对 RS 图像尤为重要。这反过来又会导致产生模糊的 SR 结果。为了解决上述问题,我们提出了尺度频率双调制网络(SFMNet)作为实现 RS 图像连续 SR 的一种手段。首先,尺度调制特征融合可以根据不同的尺度因子调制不同层次的特征融合,从而将尺度信息充分融入网络的特征提取过程。其次,频率调制重构可以在图像重构过程中调制频域信息,从而增强网络学习高频信息的能力。实验结果表明,所提出的 SFMNet 在定量指标和视觉质量方面优于现有的 RS 图像连续 SR 方法。
Scale-Frequency Dual-Modulation Method for Remote Sensing Image Continuous Super-Resolution
In recent years, the development of continuous-scale super-resolution (SR) methods in the field of remote sensing (RS) has garnered significant attention. These innovative methods are capable of delivering arbitrary-scale image SR through a single unified network. However, the majority of these methods employ the same feature extractor for different SR scales, which constrains the enhancement of network performance. Furthermore, the utilization of a multilayer perceptron for image reconstruction results in the loss of a substantial quantity of high-frequency information, which is of particular significance in the context of RS images. This, in turn, gives rise to the generation of blurred SR results. In order to address the aforementioned issues, the scale-frequency dual-modulation network (SFMNet) is proposed as a means of achieving RS image continuous SR. First, scale modulation feature fusion can modulate different levels of feature fusion according to different scale factors, thereby fully integrating the scale information into the feature extraction process of the network. Subsequently, frequency modulation reconstruction can modulate the frequency-domain information at the root of the image reconstruction process, thereby enhancing the ability of the network to learn high-frequency information. The experimental results demonstrate that the proposed SFMNet outperforms existing RS image continuous SR methods in terms of quantitative indices and visual quality.
期刊介绍:
The IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing addresses the growing field of applications in Earth observations and remote sensing, and also provides a venue for the rapidly expanding special issues that are being sponsored by the IEEE Geosciences and Remote Sensing Society. The journal draws upon the experience of the highly successful “IEEE Transactions on Geoscience and Remote Sensing” and provide a complementary medium for the wide range of topics in applied earth observations. The ‘Applications’ areas encompasses the societal benefit areas of the Global Earth Observations Systems of Systems (GEOSS) program. Through deliberations over two years, ministers from 50 countries agreed to identify nine areas where Earth observation could positively impact the quality of life and health of their respective countries. Some of these are areas not traditionally addressed in the IEEE context. These include biodiversity, health and climate. Yet it is the skill sets of IEEE members, in areas such as observations, communications, computers, signal processing, standards and ocean engineering, that form the technical underpinnings of GEOSS. Thus, the Journal attracts a broad range of interests that serves both present members in new ways and expands the IEEE visibility into new areas.