Remote sensing (RS) image-text retrieval is a practical and challenging task that has received considerable attention. Currently, most approaches rely on either convolutional neural networks or Transformers, which cannot effectively extract both global and fine-grained features simultaneously. Furthermore, the problem of high intramodal similarity in the RS domain poses a challenge for feature learning. In addition, the characteristics of model training at different stages seem to be neglected in most studies. In order to tackle these problems, we propose a fine-grained information supplementation (FGIS) and value-guided learning model that leverages prior knowledge in the RS domain for feature supplementation and employs a value-guided training approach to learn fine-grained, expressive, and robust feature representations. Specifically, we introduce the FGIS module to facilitate the supplementation of fine-grained visual features, thereby enhancing perceptual abilities for both global and local features. Furthermore, we mitigate the problem of high intra-modal similarity by proposing two loss functions: the weighted contrastive loss and the scene-adaptive fine-grained perceptual loss. Finally, we design a value-guided learning framework that focuses on the most important information at each stage of training. Extensive experiments on the remote sensing image captioning dataset (RSICD) and remote sensing image text match dataset (RSITMD) datasets verify the effectiveness and superiority of our model.
{"title":"Fine-Grained Information Supplementation and Value-Guided Learning for Remote Sensing Image-Text Retrieval","authors":"Zihui Zhou;Yong Feng;Agen Qiu;Guofan Duan;Mingliang Zhou","doi":"10.1109/JSTARS.2024.3480014","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3480014","url":null,"abstract":"Remote sensing (RS) image-text retrieval is a practical and challenging task that has received considerable attention. Currently, most approaches rely on either convolutional neural networks or Transformers, which cannot effectively extract both global and fine-grained features simultaneously. Furthermore, the problem of high intramodal similarity in the RS domain poses a challenge for feature learning. In addition, the characteristics of model training at different stages seem to be neglected in most studies. In order to tackle these problems, we propose a fine-grained information supplementation (FGIS) and value-guided learning model that leverages prior knowledge in the RS domain for feature supplementation and employs a value-guided training approach to learn fine-grained, expressive, and robust feature representations. Specifically, we introduce the FGIS module to facilitate the supplementation of fine-grained visual features, thereby enhancing perceptual abilities for both global and local features. Furthermore, we mitigate the problem of high intra-modal similarity by proposing two loss functions: the weighted contrastive loss and the scene-adaptive fine-grained perceptual loss. Finally, we design a value-guided learning framework that focuses on the most important information at each stage of training. Extensive experiments on the remote sensing image captioning dataset (RSICD) and remote sensing image text match dataset (RSITMD) datasets verify the effectiveness and superiority of our model.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19194-19210"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10716520","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142550542","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The conversion of agricultural lands, termed “nonagriculturalization,” poses profound threats to food security and ecological stability. Remote sensing image change detection offers an invaluable tool for monitoring this phenomenon. However, most change detection techniques prioritize image comparison over exploiting accumulated vector datasets. Additionally, many current methods are not readily applicable in practical scenarios due to inadequate model generalization capabilities and a scarcity of samples, resulting in a continued reliance on manual intervention for nonagriculturalization detection. In response, this article introduces a novel change detection approach for nonagriculturalization based on the vector data and contrastive learning. Initially, the boundary-constrained simple noniterative clustering algorithm is applied to segment two-phase images under vector data guidance. Samples are then generated using an adaptive cropping method. For early phase image samples, a collaborative validation-based sample annotation framework is employed to optimize and annotate the samples, with the purified high-quality samples serving as the training set for subsequent classification. For later-phase image samples, only those within the cropland vector polygons are retained for prediction. Building on this, a semi-supervised cross-domain contrastive learning framework is proposed for remote sensing scene classification. Ultimately, by integrating nonagriculturalization rules and postprocessing techniques, areas undergoing nonagriculturalization are further detected. Validating our methodology on Wuxi and Yangzhou datasets yielded precision rates of 91.57% and 89.21%, with recall rates of 93.68% and 90.51%, respectively. These outcomes affirm the effectiveness of our method in nonagriculturalization detection, offering robust technical support for research in this domain.
{"title":"Nonagriculturalization Detection Based on Vector Polygons and Contrastive Learning With High-Resolution Remote Sensing Images","authors":"Hui Zhang;Wei Liu;Changming Zhu;Hao Niu;Pengcheng Yin;Shiling Dong;Jialin Wu;Erzhu Li;Lianpeng Zhang","doi":"10.1109/JSTARS.2024.3476131","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3476131","url":null,"abstract":"The conversion of agricultural lands, termed “nonagriculturalization,” poses profound threats to food security and ecological stability. Remote sensing image change detection offers an invaluable tool for monitoring this phenomenon. However, most change detection techniques prioritize image comparison over exploiting accumulated vector datasets. Additionally, many current methods are not readily applicable in practical scenarios due to inadequate model generalization capabilities and a scarcity of samples, resulting in a continued reliance on manual intervention for nonagriculturalization detection. In response, this article introduces a novel change detection approach for nonagriculturalization based on the vector data and contrastive learning. Initially, the boundary-constrained simple noniterative clustering algorithm is applied to segment two-phase images under vector data guidance. Samples are then generated using an adaptive cropping method. For early phase image samples, a collaborative validation-based sample annotation framework is employed to optimize and annotate the samples, with the purified high-quality samples serving as the training set for subsequent classification. For later-phase image samples, only those within the cropland vector polygons are retained for prediction. Building on this, a semi-supervised cross-domain contrastive learning framework is proposed for remote sensing scene classification. Ultimately, by integrating nonagriculturalization rules and postprocessing techniques, areas undergoing nonagriculturalization are further detected. Validating our methodology on Wuxi and Yangzhou datasets yielded precision rates of 91.57% and 89.21%, with recall rates of 93.68% and 90.51%, respectively. These outcomes affirm the effectiveness of our method in nonagriculturalization detection, offering robust technical support for research in this domain.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"18474-18488"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10716558","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142517955","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-14DOI: 10.1109/JSTARS.2024.3480816
Sergi Aliaga;Marco Lanzetti;Vitaly Petrov;Anna Vizziello;Paolo Gamba;Josep M. Jornet
The integration of sub-Terahertz (sub-THz) communication beyond $mathbf {100 }text{ GHz}$ with differential absorption radar (DAR) as part of the evolution toward 5G-sdvanced and 6G nonterrestrial networks (NTNs) and beyond is critical for enabling intrinsic coexistence between these technologies. This study presents the first comprehensive analysis of an integrated sensing and communication (ISAC) system that combines satellite-centric sub-THz communications with DAR. We propose adapting the DAR waveform to be compatible with communication modulation, mathematically proving that this integration does not compromise DAR's sensing capabilities. In addition, we explore two methods to increase communication throughput with minimal impact on sensing performance: increasing the modulation order and increasing the number of symbols per chirp pulse. The results, validated through extensive simulations using published atmospheric models from the International Telecommunication Union (ITU) and the high resolution transmission molecular absorption database, reveal significant system tradeoffs. Our findings demonstrate that data rates can be enhanced up to 500 times without substantial degradation in estimation accuracy. However, excessively high data rates lead to significant estimation errors in the sensing system. This research underscores the potential of sub-THz ISAC systems for advanced satellite communications and remote sensing applications.
作为向 5G-sdvanced 和 6G nonterrestrial networks (NTNs) 及更高演进的一部分，超 $mathbf {100 }text{ GHz}$ 亚太赫兹（sub-THz）通信与差分吸收雷达（DAR）的集成对于实现这些技术之间的内在共存至关重要。本研究首次对综合传感与通信（ISAC）系统进行了全面分析，该系统将以卫星为中心的次 THz 通信与 DAR 相结合。我们建议调整 DAR 波形，使其与通信调制兼容，并用数学方法证明这种整合不会损害 DAR 的传感能力。此外，我们还探索了两种在对传感性能影响最小的情况下提高通信吞吐量的方法：增加调制阶数和增加每个啁啾脉冲的符号数。我们利用国际电信联盟（ITU）公布的大气模型和高分辨率传输分子吸收数据库进行了大量模拟，验证了这些结果，揭示了重要的系统权衡。我们的研究结果表明，数据传输速率最多可提高 500 倍，而估计精度不会大幅下降。然而，过高的数据传输率会导致传感系统出现严重的估计误差。这项研究强调了亚 THz ISAC 系统在先进卫星通信和遥感应用方面的潜力。
{"title":"Analysis of Integrated Differential Absorption Radar and Subterahertz Satellite Communications Beyond 6G","authors":"Sergi Aliaga;Marco Lanzetti;Vitaly Petrov;Anna Vizziello;Paolo Gamba;Josep M. Jornet","doi":"10.1109/JSTARS.2024.3480816","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3480816","url":null,"abstract":"The integration of sub-Terahertz (sub-THz) communication beyond \u0000<inline-formula><tex-math>$mathbf {100 }text{ GHz}$</tex-math></inline-formula>\u0000 with differential absorption radar (DAR) as part of the evolution toward 5G-sdvanced and 6G nonterrestrial networks (NTNs) and beyond is critical for enabling intrinsic coexistence between these technologies. This study presents the first comprehensive analysis of an integrated sensing and communication (ISAC) system that combines satellite-centric sub-THz communications with DAR. We propose adapting the DAR waveform to be compatible with communication modulation, mathematically proving that this integration does not compromise DAR's sensing capabilities. In addition, we explore two methods to increase communication throughput with minimal impact on sensing performance: increasing the modulation order and increasing the number of symbols per chirp pulse. The results, validated through extensive simulations using published atmospheric models from the International Telecommunication Union (ITU) and the high resolution transmission molecular absorption database, reveal significant system tradeoffs. Our findings demonstrate that data rates can be enhanced up to 500 times without substantial degradation in estimation accuracy. However, excessively high data rates lead to significant estimation errors in the sensing system. This research underscores the potential of sub-THz ISAC systems for advanced satellite communications and remote sensing applications.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19243-19259"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10716463","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142555093","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-14DOI: 10.1109/JSTARS.2024.3476175
Yuhan Kang;Jie Wu;Qiang Liu;Jun Yue;Leyuan Fang
Domain adaptation has been demonstrated to be an important technique to reduce the expensive annotation costs for remote sensing segmentation. However, for remote sensing images (RSIs) acquired from different imaging modalities with significant differences, a model trained on one modality can hardly be utilized for images of other modalities. This leads to a greater challenge in domain adaptation, called heterogeneous domain adaptation (HDA). To address this issue, we propose a novel method called transfer diffusion (Trans-Diff), which is the first work to explore the diffusion model for HDA remote sensing segmentation. The proposed Trans-Diff constructs cross-domain unified prompts for the diffusion model. This approach enables the generation of images from different modalities with specific semantics, leading to efficient HDA segmentation. Specifically, we first propose an interrelated semantic modeling method to establish semantic interrelation between heterogeneous RSIs and annotations in a high-dimensional feature space and extract the unified features as the cross-domain prompts. Then, we construct a semantic guidance diffusion model to further improve the semantic guidance of images generated with the cross-domain prompts, which effectively facilitates the semantic transfer of RSIs from source modality to target modality. In addition, we design an adaptive sampling strategy to dynamically regulate the generated images' stylistic consistency and semantic consistency. This can effectively reduce the cross-domain discrepancies between different modalities of RSIs, ultimately significantly improving the HDA remote sensing segmentation performance. Experimental results demonstrate the superior performance of Trans-Diff over advanced methods on several heterogeneous RSI datasets.
{"title":"Trans-Diff: Heterogeneous Domain Adaptation for Remote Sensing Segmentation With Transfer Diffusion","authors":"Yuhan Kang;Jie Wu;Qiang Liu;Jun Yue;Leyuan Fang","doi":"10.1109/JSTARS.2024.3476175","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3476175","url":null,"abstract":"Domain adaptation has been demonstrated to be an important technique to reduce the expensive annotation costs for remote sensing segmentation. However, for remote sensing images (RSIs) acquired from different imaging modalities with significant differences, a model trained on one modality can hardly be utilized for images of other modalities. This leads to a greater challenge in domain adaptation, called heterogeneous domain adaptation (HDA). To address this issue, we propose a novel method called transfer diffusion (Trans-Diff), which is the first work to explore the diffusion model for HDA remote sensing segmentation. The proposed Trans-Diff constructs cross-domain unified prompts for the diffusion model. This approach enables the generation of images from different modalities with specific semantics, leading to efficient HDA segmentation. Specifically, we first propose an interrelated semantic modeling method to establish semantic interrelation between heterogeneous RSIs and annotations in a high-dimensional feature space and extract the unified features as the cross-domain prompts. Then, we construct a semantic guidance diffusion model to further improve the semantic guidance of images generated with the cross-domain prompts, which effectively facilitates the semantic transfer of RSIs from source modality to target modality. In addition, we design an adaptive sampling strategy to dynamically regulate the generated images' stylistic consistency and semantic consistency. This can effectively reduce the cross-domain discrepancies between different modalities of RSIs, ultimately significantly improving the HDA remote sensing segmentation performance. Experimental results demonstrate the superior performance of Trans-Diff over advanced methods on several heterogeneous RSI datasets.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"18413-18426"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10716600","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142517986","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-14DOI: 10.1109/JSTARS.2024.3479703
Zhiyuan Ji;Xueqian Wang;Zhihao Wang;Gang Li
In this article, we consider the issue of change detection (CD) for heterogeneous remote sensing images. Existing deep learning-based methods for CD usually utilize square convolution receptive fields, which do not sufficiently exploit the contextual and boundary information in heterogeneous images. To address the aforementioned issue, we propose an enhanced and unsupervised Siamese superpixel-based network for CD in heterogeneous remote sensing images. Our newly proposed method innovatively combines superpixels with the square receptive fields to generate the boundary adherence receptive fields and better capture the contextual information than existing methods only with the regular square receptive fields. Furthermore, we utilize an adaptive superpixel merging module to prevent the oversegmentation of superpixels and strengthen the robustness of our method in terms of superpixel sizes. Experiments based on four real datasets demonstrate that the proposed method achieves higher accuracy than other commonly used CD methods in heterogeneous remote sensing images.
本文探讨了异质遥感图像的变化检测（CD）问题。现有的基于深度学习的变化检测方法通常使用方形卷积感受野，但这种方法不能充分利用异构图像中的上下文和边界信息。为解决上述问题，我们提出了一种基于暹罗超像素的增强型无监督网络，用于异构遥感图像的 CD。我们新提出的方法创新性地将超像素与方形感受野结合起来，生成边界粘附感受野，与现有的仅使用常规方形感受野的方法相比，能更好地捕捉上下文信息。此外，我们还利用自适应超像素合并模块来防止超像素的过度分割，并加强了我们的方法在超像素大小方面的鲁棒性。基于四个真实数据集的实验证明，在异质遥感图像中，所提出的方法比其他常用的 CD 方法获得了更高的精度。
{"title":"An Enhanced and Unsupervised Siamese Network With Superpixel-Guided Learning for Change Detection in Heterogeneous Remote Sensing Images","authors":"Zhiyuan Ji;Xueqian Wang;Zhihao Wang;Gang Li","doi":"10.1109/JSTARS.2024.3479703","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3479703","url":null,"abstract":"In this article, we consider the issue of change detection (CD) for heterogeneous remote sensing images. Existing deep learning-based methods for CD usually utilize square convolution receptive fields, which do not sufficiently exploit the contextual and boundary information in heterogeneous images. To address the aforementioned issue, we propose an enhanced and unsupervised Siamese superpixel-based network for CD in heterogeneous remote sensing images. Our newly proposed method innovatively combines superpixels with the square receptive fields to generate the boundary adherence receptive fields and better capture the contextual information than existing methods only with the regular square receptive fields. Furthermore, we utilize an adaptive superpixel merging module to prevent the oversegmentation of superpixels and strengthen the robustness of our method in terms of superpixel sizes. Experiments based on four real datasets demonstrate that the proposed method achieves higher accuracy than other commonly used CD methods in heterogeneous remote sensing images.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19451-19466"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10715669","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142595066","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-14DOI: 10.1109/JSTARS.2024.3479212
Guanglian Zhang;Lan Zhang;Zhanxu Zhang;Jiangwei Deng;Lifeng Bian;Chen Yang
Methods for joint classification of hyperspectral images (HSIs) with high dimensionality and spectral correlation and other sensor data (e.g., optical, infrared, radar, etc.) are important directions in the field of remote sensing. To better learn the feature representation of diffusion features (HSI), the unsupervised global modeling property of diffusion is utilized to mine the potential features of HSI to obtain diffusion features as input data. In addition, to fuse HSI features, HSI diffusion features, and other data features, a three-input diffusion-enhanced CNN–transformer (DECT) network based on CNN and transformer is proposed for feature extraction and fusion. First, the primary features are extracted by hierarchical CNN after premodal fusion. Second, considering the high dimensionality of HSI, spectral pooling attention interaction is designed for feature extraction and aggregation of information from different attentions. Finally, the inverted bottleneck convolutional transformer is designed to aggregate multisource information to enhance feature reuse and aggregate local and contextual information. It is shown on three publicly available datasets that DECT outperforms current state-of-the-art methods.
{"title":"DECT: Diffusion-Enhanced CNN–Transformer for Multisource Remote Sensing Data Classification","authors":"Guanglian Zhang;Lan Zhang;Zhanxu Zhang;Jiangwei Deng;Lifeng Bian;Chen Yang","doi":"10.1109/JSTARS.2024.3479212","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3479212","url":null,"abstract":"Methods for joint classification of hyperspectral images (HSIs) with high dimensionality and spectral correlation and other sensor data (e.g., optical, infrared, radar, etc.) are important directions in the field of remote sensing. To better learn the feature representation of diffusion features (HSI), the unsupervised global modeling property of diffusion is utilized to mine the potential features of HSI to obtain diffusion features as input data. In addition, to fuse HSI features, HSI diffusion features, and other data features, a three-input diffusion-enhanced CNN–transformer (DECT) network based on CNN and transformer is proposed for feature extraction and fusion. First, the primary features are extracted by hierarchical CNN after premodal fusion. Second, considering the high dimensionality of HSI, spectral pooling attention interaction is designed for feature extraction and aggregation of information from different attentions. Finally, the inverted bottleneck convolutional transformer is designed to aggregate multisource information to enhance feature reuse and aggregate local and contextual information. It is shown on three publicly available datasets that DECT outperforms current state-of-the-art methods.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19288-19301"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10716525","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142555103","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-14DOI: 10.1109/JSTARS.2024.3479920
Chhaya Katiyar;Vidya Manian
Hyperspectral image (HSI) change detection focuses on identifying differences in multitemporal HSIs. Graph convolutional networks (GCNs) have demonstrated greater promise than convolutional neural networks (CNNs) in remote sensing, particularly for processing HSIs. This is due to GCN's ability to handle non-Euclidean graph-structured information, as opposed to the fixed kernel operations of CNN based on Euclidean structures. Specifically, GCN operates predominantly on superpixel-based nodes. This article proposes a method, named hyperspectral multilevel GCN and CNN feature fusion (HMGCF) for change detection, that integrates superpixel-level GCN with pixel-level CNN for feature extraction and efficient change detection in HSI. The proposed method utilizes the strengths of both CNN and GCN; the CNN branch focuses on feature learning in small-scale, regular regions, while the GCN branch handles large-scale, irregular regions. This approach generates complementary spectral–spatial features at both pixel and superpixel levels. To bridge the structural incompatibility between the Euclidean-data-oriented CNN and the non-Euclidean-data-oriented GCN, HMGCF introduces a graph encoder and decoder. These elements help in propagating features between image pixels and graph nodes, allowing CNN and GCN to function within an integrated end-to-end framework. HMGCF integrates graph encoding into the network, edge weights, and node representations from training data. Ablation studies on four datasets reveal that the combination of CNN and GCN branches in the HMGCF model consistently outperforms existing methods by margins ranging from 0.5% to 2.5%. In addition, HMGCF shows significant improvements in both kappa and $F1$ scores in all datasets.
{"title":"Hyperspectral Multilevel GCN and CNN Feature Fusion for Change Detection","authors":"Chhaya Katiyar;Vidya Manian","doi":"10.1109/JSTARS.2024.3479920","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3479920","url":null,"abstract":"Hyperspectral image (HSI) change detection focuses on identifying differences in multitemporal HSIs. Graph convolutional networks (GCNs) have demonstrated greater promise than convolutional neural networks (CNNs) in remote sensing, particularly for processing HSIs. This is due to GCN's ability to handle non-Euclidean graph-structured information, as opposed to the fixed kernel operations of CNN based on Euclidean structures. Specifically, GCN operates predominantly on superpixel-based nodes. This article proposes a method, named hyperspectral multilevel GCN and CNN feature fusion (HMGCF) for change detection, that integrates superpixel-level GCN with pixel-level CNN for feature extraction and efficient change detection in HSI. The proposed method utilizes the strengths of both CNN and GCN; the CNN branch focuses on feature learning in small-scale, regular regions, while the GCN branch handles large-scale, irregular regions. This approach generates complementary spectral–spatial features at both pixel and superpixel levels. To bridge the structural incompatibility between the Euclidean-data-oriented CNN and the non-Euclidean-data-oriented GCN, HMGCF introduces a graph encoder and decoder. These elements help in propagating features between image pixels and graph nodes, allowing CNN and GCN to function within an integrated end-to-end framework. HMGCF integrates graph encoding into the network, edge weights, and node representations from training data. Ablation studies on four datasets reveal that the combination of CNN and GCN branches in the HMGCF model consistently outperforms existing methods by margins ranging from 0.5% to 2.5%. In addition, HMGCF shows significant improvements in both kappa and \u0000<inline-formula><tex-math>$F1$</tex-math></inline-formula>\u0000 scores in all datasets.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19146-19159"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10715660","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524156","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-14DOI: 10.1109/JSTARS.2024.3479315
Cheng Wu;Qifan Yu;Shaoning Li;Anmin Fu;Mengguang Liao;Lelin Li
The horizontal positioning error in spaceborne photon point clouds seriously constrains their elevation accuracy. To improve data quality for enhanced performance in scientific applications, this study proposes a photon correction method based on terrain feature identification, specifically for the photon-counting spaceborne lidar. Unlike the conventional terrain matching method, this approach accurately determines the horizontal positions of photons within a small-range area by establishing a matching relationship between the laser elevation turning points and the surface boundary lines. The feasibility of this method was verified using the satellite laser altimetry simulation platform, and the horizontal correction accuracy can reach within 0.6 m. Subsequently, the experiments were conducted to verify the geometric positioning accuracy of ICESat-2 across different areas, leveraging high-precision digital surface models. The results indicate that the average horizontal accuracy of ICESat-2 was 3.81 m, and the elevation accuracy was better than 0.5 m.
{"title":"Geometric Positioning Verification of Spaceborne Photon-Counting Lidar Data Based on Terrain Feature Identification","authors":"Cheng Wu;Qifan Yu;Shaoning Li;Anmin Fu;Mengguang Liao;Lelin Li","doi":"10.1109/JSTARS.2024.3479315","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3479315","url":null,"abstract":"The horizontal positioning error in spaceborne photon point clouds seriously constrains their elevation accuracy. To improve data quality for enhanced performance in scientific applications, this study proposes a photon correction method based on terrain feature identification, specifically for the photon-counting spaceborne lidar. Unlike the conventional terrain matching method, this approach accurately determines the horizontal positions of photons within a small-range area by establishing a matching relationship between the laser elevation turning points and the surface boundary lines. The feasibility of this method was verified using the satellite laser altimetry simulation platform, and the horizontal correction accuracy can reach within 0.6 m. Subsequently, the experiments were conducted to verify the geometric positioning accuracy of ICESat-2 across different areas, leveraging high-precision digital surface models. The results indicate that the average horizontal accuracy of ICESat-2 was 3.81 m, and the elevation accuracy was better than 0.5 m.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19408-19419"},"PeriodicalIF":4.7,"publicationDate":"2024-10-14","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10715570","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142579243","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2024-10-11DOI: 10.1109/JSTARS.2024.3478350
Yonghong Liu;Fuzhong Weng;Fei Tang;Rui Li;Yongming Xu;Yang Han;Jun Yang;Qingyang Liu
Accurate information on microwave land surface emissivity (MLSE) is important for satellite data assimilation. In this article, a new random forest (RF) algorithm is developed for retrieving MLSE under all-sky conditions. Using Level-1 brightness temperature data from the FengYun-3D (FY-3D) microwave radiation imager in 2022, two global MLSE daily product datasets, clear-sky (FY-3D1) and clear/cloudy (FY-3D2), were obtained by using one-dimensional variational method and microwave radiative transfer method, respectively. Based on the global spatiotemporal consistency assessment, a high-quality daily MLSE training dataset for the Tibetan Plateau was selected from the two datasets. Then, ten land surface parameters from routine observation were chosen as input features to the RF model to simulate the MLSE under all-sky conditions in the Tibetan Plateau. The results show that both FY-3D1 and FY-3D2 MLSE datasets are comparable to the international mainstream MLSE products in quality, while the clear sky FY-3D1 is likely to be better than the clear/cloudy FY-3D2 MLSE. Land surface roughness, vegetation optical thickness, normalized vegetation index, and land cover type are identified as the most important factors affecting MLSE in the Tibetan Plateau. The RF model can effectively simulate the MLSE in the frequency range of 10.65–89.0 GHz under all-sky conditions. The coefficients of determination ( R 2 ) for horizontal polarization and vertical polarization range from 0.86 (10.65 GHz) to 0.91 (18.7 GHz) and from 0.60 (10.65 GHz) to 0.74 (89.0 GHz), respectively. The root mean square errors for horizontal polarization and vertical polarization range from 0.017 (23.8 GHz) to 0.023 (10.65 GHz) and from 0.016 (10.65 GHz) to 0.019 (89.0 GHz), respectively. These results indicate that machine learning is likely to be an effective method for future all-sky simulation of MLSE.
{"title":"Simulations of Microwave Land Surface Emissivity Using FengYun-3D Microwave Radiation Imager Data: A Case in the Tibetan Plateau","authors":"Yonghong Liu;Fuzhong Weng;Fei Tang;Rui Li;Yongming Xu;Yang Han;Jun Yang;Qingyang Liu","doi":"10.1109/JSTARS.2024.3478350","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3478350","url":null,"abstract":"Accurate information on microwave land surface emissivity (MLSE) is important for satellite data assimilation. In this article, a new random forest (RF) algorithm is developed for retrieving MLSE under all-sky conditions. Using Level-1 brightness temperature data from the FengYun-3D (FY-3D) microwave radiation imager in 2022, two global MLSE daily product datasets, clear-sky (FY-3D1) and clear/cloudy (FY-3D2), were obtained by using one-dimensional variational method and microwave radiative transfer method, respectively. Based on the global spatiotemporal consistency assessment, a high-quality daily MLSE training dataset for the Tibetan Plateau was selected from the two datasets. Then, ten land surface parameters from routine observation were chosen as input features to the RF model to simulate the MLSE under all-sky conditions in the Tibetan Plateau. The results show that both FY-3D1 and FY-3D2 MLSE datasets are comparable to the international mainstream MLSE products in quality, while the clear sky FY-3D1 is likely to be better than the clear/cloudy FY-3D2 MLSE. Land surface roughness, vegetation optical thickness, normalized vegetation index, and land cover type are identified as the most important factors affecting MLSE in the Tibetan Plateau. The RF model can effectively simulate the MLSE in the frequency range of 10.65–89.0 GHz under all-sky conditions. The coefficients of determination (\u0000<italic>R</i>\u0000<sup>2</sup>\u0000) for horizontal polarization and vertical polarization range from 0.86 (10.65 GHz) to 0.91 (18.7 GHz) and from 0.60 (10.65 GHz) to 0.74 (89.0 GHz), respectively. The root mean square errors for horizontal polarization and vertical polarization range from 0.017 (23.8 GHz) to 0.023 (10.65 GHz) and from 0.016 (10.65 GHz) to 0.019 (89.0 GHz), respectively. These results indicate that machine learning is likely to be an effective method for future all-sky simulation of MLSE.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19078-19094"},"PeriodicalIF":4.7,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10714024","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142524196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Synthetic aperture radar (SAR) imaging is hindered by coherent imaging mechanisms, leading to degradation in image quality due to speckle. Current methods can effectively suppress speckles but may lead to varying degrees of edge information loss. The accurate and comprehensive evaluation of speckle suppression is vital for enhancing SAR image interpretation. Therefore, this study proposes a novel method for evaluating despeckling image quality based on adaptively extracting homogeneity and heterogeneity scene patches (IQE_HHSP). The proposed IQE_HHSP effectively extracts homogeneous and heterogeneous patches, and performs speckle suppression and edge-preservation evaluation based on the extracted patch feature, achieving comprehensive evaluation of filtered image. The efficacy of IQE_HHSP is demonstrated through the evaluation on four SAR images using six comparative evaluation indicators. The experimental results indicate that IQE_HHSP provides the accurate quality assessment of SAR filtering models, yielding results consistent with visual observations.
合成孔径雷达（SAR）成像受到相干成像机制的阻碍，斑点会导致图像质量下降。目前的方法可以有效抑制斑点，但可能会导致不同程度的边缘信息丢失。准确、全面地评估斑点抑制对提高合成孔径雷达图像判读至关重要。因此，本研究提出了一种基于自适应提取同质和异质场景斑块（IQE_HHSP）的新型消斑图像质量评估方法。所提出的 IQE_HHSP 能有效提取同质和异质斑块，并根据提取的斑块特征进行斑点抑制和边缘保留评价，实现对滤波图像的综合评价。通过在四幅合成孔径雷达图像上使用六项比较评价指标进行评估，证明了 IQE_HHSP 的功效。实验结果表明，IQE_HHSP 可对 SAR 滤波模型进行准确的质量评估，得出的结果与目测结果一致。
{"title":"Despeckling SAR Image Quality Evaluation by Homogeneity and Heterogeneity Scene Patches","authors":"Chuang Sun;Fengcheng Guo;Zhaoling Hu;Lianpeng Zhang;Wensong Liu;Tingting Huang","doi":"10.1109/JSTARS.2024.3479229","DOIUrl":"https://doi.org/10.1109/JSTARS.2024.3479229","url":null,"abstract":"Synthetic aperture radar (SAR) imaging is hindered by coherent imaging mechanisms, leading to degradation in image quality due to speckle. Current methods can effectively suppress speckles but may lead to varying degrees of edge information loss. The accurate and comprehensive evaluation of speckle suppression is vital for enhancing SAR image interpretation. Therefore, this study proposes a novel method for evaluating despeckling image quality based on adaptively extracting homogeneity and heterogeneity scene patches (IQE_HHSP). The proposed IQE_HHSP effectively extracts homogeneous and heterogeneous patches, and performs speckle suppression and edge-preservation evaluation based on the extracted patch feature, achieving comprehensive evaluation of filtered image. The efficacy of IQE_HHSP is demonstrated through the evaluation on four SAR images using six comparative evaluation indicators. The experimental results indicate that IQE_HHSP provides the accurate quality assessment of SAR filtering models, yielding results consistent with visual observations.","PeriodicalId":13116,"journal":{"name":"IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing","volume":"17 ","pages":"19211-19229"},"PeriodicalIF":4.7,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10715487","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142555094","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":2,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：