首页 > 最新文献

ISPRS Journal of Photogrammetry and Remote Sensing最新文献

英文 中文
Global perspectives on sand dune patterns: Scale-adaptable classification using Landsat imagery and deep learning strategies 沙丘模式的全球视角:利用大地遥感卫星图像和深度学习策略进行规模适应性分类
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-15 DOI: 10.1016/j.isprsjprs.2024.10.002
Zhijia Zheng , Xiuyuan Zhang , Jiajun Li , Eslam Ali , Jinsongdi Yu , Shihong Du
Sand dune patterns (SDPs) are spatial aggregations of dunes and interdunes, exhibiting distinct morphologies and spatial structures. Recognizing global SDPs is crucial for understanding the development processes, contributing factors, and self-organization characteristics of aeolian systems. However, the diversity, complexity, and multiscale nature of global SDPs poses significant technical challenges in the classification scheme, sample collection, feature representation, and classification method. This study addresses these challenges by developing a novel global SDP classification approach based on an advanced deep-learning network. Firstly, we established a globally applicable SDP classification scheme that accommodates the diversity nature of SDPs. Secondly, we developed an SDP semantic segmentation sample dataset, which encompassed a wide array of SDP representations. Thirdly, we deployed the SegFormer network to automatically capture detailed dune structures and developed a weighted voting strategy to ensure scale adaptability. Experiments utilizing Landsat-8 imagery yielded a commendable overall accuracy (OA) of 85.43 %. Notably, most SDP types exhibited high classification accuracies, such as star dunes (97.43 %) and simple linear dunes (87.17 %). The weighted voting strategy prioritized the predictions of each type, resulting in a 1.41 %∼7.91 % improvement in OA compared to the single-scale classification and average voting methods. This innovative approach facilitated the generation of a high-quality, fine-grained, and global-scale SDP map at 30 m resolution (GSDP30), which not only directly provides the spatial distribution of global SDPs but also serves as valuable support for understanding aeolian processes. This study represents the first instance of producing such a comprehensive and globally applicable SDP map at this fine resolution.
沙丘模式(SDPs)是沙丘和沙丘间的空间聚集,表现出独特的形态和空间结构。认识全球沙丘形态对于了解风化系统的发展过程、成因和自组织特征至关重要。然而,全球 SDP 的多样性、复杂性和多尺度性在分类方案、样本收集、特征表示和分类方法等方面带来了巨大的技术挑战。本研究通过开发一种基于先进深度学习网络的新型全局 SDP 分类方法来应对这些挑战。首先,我们建立了一种全球适用的 SDP 分类方案,该方案考虑到了 SDP 的多样性。其次,我们开发了一个 SDP 语义分割样本数据集,其中包含了大量 SDP 表征。第三,我们部署了 SegFormer 网络来自动捕捉沙丘的详细结构,并开发了一种加权投票策略来确保规模适应性。利用 Landsat-8 图像进行的实验取得了令人称道的 85.43% 的总体准确率 (OA)。值得注意的是,大多数 SDP 类型都表现出较高的分类准确率,如星形沙丘(97.43%)和简单线性沙丘(87.17%)。加权投票策略对每种类型的预测进行了优先排序,与单尺度分类法和平均投票法相比,OA 提高了 1.41 %∼7.91 %。这种创新方法有助于生成 30 米分辨率的高质量、细粒度和全球尺度 SDP 地图(GSDP30),它不仅直接提供了全球 SDP 的空间分布情况,还为了解风化过程提供了宝贵的支持。这项研究是首次以如此精细的分辨率绘制如此全面和全球适用的 SDP 地图。
{"title":"Global perspectives on sand dune patterns: Scale-adaptable classification using Landsat imagery and deep learning strategies","authors":"Zhijia Zheng ,&nbsp;Xiuyuan Zhang ,&nbsp;Jiajun Li ,&nbsp;Eslam Ali ,&nbsp;Jinsongdi Yu ,&nbsp;Shihong Du","doi":"10.1016/j.isprsjprs.2024.10.002","DOIUrl":"10.1016/j.isprsjprs.2024.10.002","url":null,"abstract":"<div><div>Sand dune patterns (SDPs) are spatial aggregations of dunes and interdunes, exhibiting distinct morphologies and spatial structures. Recognizing global SDPs is crucial for understanding the development processes, contributing factors, and self-organization characteristics of aeolian systems. However, the diversity, complexity, and multiscale nature of global SDPs poses significant technical challenges in the classification scheme, sample collection, feature representation, and classification method. This study addresses these challenges by developing a novel global SDP classification approach based on an advanced deep-learning network. Firstly, we established a globally applicable SDP classification scheme that accommodates the diversity nature of SDPs. Secondly, we developed an SDP semantic segmentation sample dataset, which encompassed a wide array of SDP representations. Thirdly, we deployed the SegFormer network to automatically capture detailed dune structures and developed a weighted voting strategy to ensure scale adaptability. Experiments utilizing Landsat-8 imagery yielded a commendable overall accuracy (OA) of 85.43 %. Notably, most SDP types exhibited high classification accuracies, such as star dunes (97.43 %) and simple linear dunes (87.17 %). The weighted voting strategy prioritized the predictions of each type, resulting in a 1.41 %∼7.91 % improvement in OA compared to the single-scale classification and average voting methods. This innovative approach facilitated the generation of a high-quality, fine-grained, and global-scale SDP map at 30 m resolution (GSDP30), which not only directly provides the spatial distribution of global SDPs but also serves as valuable support for understanding aeolian processes. This study represents the first instance of producing such a comprehensive and globally applicable SDP map at this fine resolution.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 781-801"},"PeriodicalIF":10.6,"publicationDate":"2024-10-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142442945","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Mineral detection based on hyperspectral remote sensing imagery on Mars: From detection methods to fine mapping 基于火星高光谱遥感图像的矿物探测:从探测方法到精细绘图
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-12 DOI: 10.1016/j.isprsjprs.2024.09.020
Tian Ke , Yanfei Zhong , Mi Song , Xinyu Wang , Liangpei Zhang
Hyperspectral remote sensing is a commonly used technical means for mineral detection on the Martian surface, which has important implications for the study of Martian geological evolution and the study for potential biological signatures. The increasing volume of Martian remote sensing data and complex issues such as the intimate mixture of Martian minerals make research on Martian mineral detection challenging. This paper summarizes the existing achievements by analyzing the papers published in recent years and looks forward to the future research directions. Specifically, this paper introduces the currently used hyperspectral remote sensing data of Mars and systematically analyzes the characteristics and distribution of Martian minerals. The existing methods are then divided into two groups, according to their core idea, i.e., methods based on pixels and methods based on subpixels. In addition, some applications of Martian mineral detection at global and local scales are analyzed. Furthermore, the various typical methods are compared using synthetic and real data to assess their performance. The conclusion is drawn that approach based on spectral unmixing is more applicable to areas with limited and unknown mineral categories than pixel-based methods. Among them, the fully autonomous hyperspectral unmixing method can improve the overall accuracy in real CRISM images and has great potential for Martian mineral detection. The development trends are analyzed from three aspects. Firstly, in terms of data, a more complete spectral library, covering more spectral information of the Martian surface minerals, should be constructed to assist with mineral detection. Secondly, in terms of methods, spectral unmixing methods based on a nonlinear mixing model and a new generation of data-driven detection paradigms guided by Mars mineral knowledge should be developed. Finally, in terms of application, the global mapping of Martian minerals toward a more intelligent, global scale, and refined direction should be targeted in the future. The data and source code in the experiment are available at http://rsidea.whu.edu.cn/Martian_mineral_detection.htm.
高光谱遥感是探测火星表面矿物的常用技术手段,对研究火星地质演变和潜在生物特征具有重要意义。火星遥感数据量的不断增加,以及火星矿物混杂等复杂问题,使得火星矿物探测研究面临挑战。本文通过分析近年来发表的论文,总结了现有成果,并展望了未来的研究方向。具体而言,本文介绍了目前使用的火星高光谱遥感数据,并系统分析了火星矿物的特征和分布。然后将现有方法按其核心思想分为两类,即基于像素的方法和基于子像素的方法。此外,还分析了火星矿物探测在全球和局部尺度上的一些应用。此外,还使用合成数据和真实数据对各种典型方法进行了比较,以评估其性能。得出的结论是,与基于像素的方法相比,基于光谱非混合的方法更适用于矿物类别有限和未知的区域。其中,完全自主的高光谱非混合方法可以提高真实 CRISM 图像的整体精度,在火星矿物探测方面具有巨大潜力。本文从三个方面分析了其发展趋势。首先,在数据方面,应构建更完整的光谱库,涵盖更多火星表面矿物的光谱信息,以辅助矿物探测。其次,在方法方面,应开发基于非线性混合模型的光谱非混合方法和以火星矿物知识为指导的新一代数据驱动探测范式。最后,在应用方面,未来应瞄准火星矿物的全球绘图,朝着更加智能化、全球尺度化和精细化的方向发展。该实验的数据和源代码可在 http://rsidea.whu.edu.cn/Martian_mineral_detection.htm 上查阅。
{"title":"Mineral detection based on hyperspectral remote sensing imagery on Mars: From detection methods to fine mapping","authors":"Tian Ke ,&nbsp;Yanfei Zhong ,&nbsp;Mi Song ,&nbsp;Xinyu Wang ,&nbsp;Liangpei Zhang","doi":"10.1016/j.isprsjprs.2024.09.020","DOIUrl":"10.1016/j.isprsjprs.2024.09.020","url":null,"abstract":"<div><div>Hyperspectral remote sensing is a commonly used technical means for mineral detection on the Martian surface, which has important implications for the study of Martian geological evolution and the study for potential biological signatures. The increasing volume of Martian remote sensing data and complex issues such as the intimate mixture of Martian minerals make research on Martian mineral detection challenging. This paper summarizes the existing achievements by analyzing the papers published in recent years and looks forward to the future research directions. Specifically, this paper introduces the currently used hyperspectral remote sensing data of Mars and systematically analyzes the characteristics and distribution of Martian minerals. The existing methods are then divided into two groups, according to their core idea, i.e., methods based on pixels and methods based on subpixels. In addition, some applications of Martian mineral detection at global and local scales are analyzed. Furthermore, the various typical methods are compared using synthetic and real data to assess their performance. The conclusion is drawn that approach based on spectral unmixing is more applicable to areas with limited and unknown mineral categories than pixel-based methods. Among them, the fully autonomous hyperspectral unmixing method can improve the overall accuracy in real CRISM images and has great potential for Martian mineral detection. The development trends are analyzed from three aspects. Firstly, in terms of data, a more complete spectral library, covering more spectral information of the Martian surface minerals, should be constructed to assist with mineral detection. Secondly, in terms of methods, spectral unmixing methods based on a nonlinear mixing model and a new generation of data-driven detection paradigms guided by Mars mineral knowledge should be developed. Finally, in terms of application, the global mapping of Martian minerals toward a more intelligent, global scale, and refined direction should be targeted in the future. The data and source code in the experiment are available at <span><span>http://rsidea.whu.edu.cn/Martian_mineral_detection.htm</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 761-780"},"PeriodicalIF":10.6,"publicationDate":"2024-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426650","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A highly efficient index for robust mapping of tidal flats from sentinel-2 images directly 从哨兵-2 图像直接绘制潮滩稳健地图的高效指数
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-12 DOI: 10.1016/j.isprsjprs.2024.10.005
Pengfei Tang , Shanchuan Guo , Peng Zhang , Lu Qie , Xiaoquan Pan , Jocelyn Chanussot , Peijun Du
As an essential component of the intertidal zone, tidal flats (TFs) are areas rich in resources where with the most intense material and energy exchanges. However, due to the dual threats of human activities and extreme climate conditions, TFs are disappearing on a large scale. Despite their importance, accurately mapping TFs has proved challenging due to their complex and dynamic nature. Nevertheless, Tidal influences significantly enhance the diversity and variability of TFs, and suspended particulates introduce turbidity that challenges conventional indices used for distinguishing between water and land. This study focuses on the world’s largest intertidal sedimentary system located along the central coast of Jiangsu, an area characterized by complex sedimentary features and dynamic TF conditions. Through quantitative analysis of the spectral characteristics of TFs at different years, seasons, and tidal stages, this study identifies two unique spectral features of TFs: uniformly low reflectance values and a trapezoidal spectral shape. Leveraging the low reflectance, the flatness of the middle segment in the trapezoidal spectral shape, and the initial increase followed by a decreasing trend across critical bands, a novel Tidal Flat Index (TFI) has been developed. Experimental results indicate that TFI is suitable for robust and direct TF mapping across years, seasons, and tidal stages, achieving F1 scores exceeding 0.95 in 12 different scenarios. Compared to other indices and rule-based methods, TFI offers greater accuracy, threshold stability, background and cloud suppression. The study also extends to other globally rich TFs regions to demonstrate the universality and applicability of the proposed index in various environments, including its effectiveness in delineating annual TFs extents. This study offers technical support for the automatic mapping of TFs based on single Sentinel-2 multispectral images.
作为潮间带的重要组成部分,滩涂(TFs)资源丰富,物质和能量交换最为频繁。然而,由于人类活动和极端气候条件的双重威胁,潮滩正在大规模消失。尽管潮汐区十分重要,但由于其复杂多变的性质,准确绘制潮汐区地图仍具有挑战性。然而,潮汐影响极大地增强了 TFs 的多样性和变异性,悬浮颗粒带来的浑浊度挑战了用于区分水域和陆地的传统指数。本研究的重点是位于江苏中部沿海的世界上最大的潮间带沉积系统,该地区具有复杂的沉积特征和动态 TF 条件。通过定量分析潮间带在不同年份、季节和潮汐阶段的光谱特征,本研究发现了潮间带的两个独特光谱特征:均匀的低反射率值和梯形光谱形状。利用梯形光谱形状中的低反射率、中间段的平坦性以及各临界波段的先增后减趋势,开发了一种新的潮汐平坦指数(TFI)。实验结果表明,TFI 适用于跨年、跨季和跨潮汐阶段的稳健而直接的 TF 映射,在 12 种不同情况下的 F1 分数均超过 0.95。与其他指数和基于规则的方法相比,TFI 具有更高的准确性、阈值稳定性、背景和云抑制能力。该研究还扩展到其他全球丰富的 TFs 区域,以证明所提出的指数在各种环境中的普遍性和适用性,包括其在划分年度 TFs 范围方面的有效性。这项研究为基于单幅哨兵-2 多光谱图像自动绘制 TFs 地图提供了技术支持。
{"title":"A highly efficient index for robust mapping of tidal flats from sentinel-2 images directly","authors":"Pengfei Tang ,&nbsp;Shanchuan Guo ,&nbsp;Peng Zhang ,&nbsp;Lu Qie ,&nbsp;Xiaoquan Pan ,&nbsp;Jocelyn Chanussot ,&nbsp;Peijun Du","doi":"10.1016/j.isprsjprs.2024.10.005","DOIUrl":"10.1016/j.isprsjprs.2024.10.005","url":null,"abstract":"<div><div>As an essential component of the intertidal zone, tidal flats (TFs) are areas rich in resources where with the most intense material and energy exchanges. However, due to the dual threats of human activities and extreme climate conditions, TFs are disappearing on a large scale. Despite their importance, accurately mapping TFs has proved challenging due to their complex and dynamic nature. Nevertheless, Tidal influences significantly enhance the diversity and variability of TFs, and suspended particulates introduce turbidity that challenges conventional indices used for distinguishing between water and land. This study focuses on the world’s largest intertidal sedimentary system located along the central coast of Jiangsu, an area characterized by complex sedimentary features and dynamic TF conditions. Through quantitative analysis of the spectral characteristics of TFs at different years, seasons, and tidal stages, this study identifies two unique spectral features of TFs: uniformly low reflectance values and a trapezoidal spectral shape. Leveraging the low reflectance, the flatness of the middle segment in the trapezoidal spectral shape, and the initial increase followed by a decreasing trend across critical bands, a novel Tidal Flat Index (TFI) has been developed. Experimental results indicate that TFI is suitable for robust and direct TF mapping across years, seasons, and tidal stages, achieving F1 scores exceeding 0.95 in 12 different scenarios. Compared to other indices and rule-based methods, TFI offers greater accuracy, threshold stability, background and cloud suppression. The study also extends to other globally rich TFs regions to demonstrate the universality and applicability of the proposed index in various environments, including its effectiveness in delineating annual TFs extents. This study offers technical support for the automatic mapping of TFs based on single Sentinel-2 multispectral images.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 742-760"},"PeriodicalIF":10.6,"publicationDate":"2024-10-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426643","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Improving drone-based uncalibrated estimates of wheat canopy temperature in plot experiments by accounting for confounding factors in a multi-view analysis 在多视角分析中考虑混杂因素,改进小区试验中基于无人机的小麦冠层温度非校准估计值
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-11 DOI: 10.1016/j.isprsjprs.2024.09.015
Simon Treier , Juan M. Herrera , Andreas Hund , Norbert Kirchgessner , Helge Aasen , Achim Walter , Lukas Roth
Canopy temperature (CT) is an integrative trait, indicative of the relative fitness of a plant genotype to the environment. Lower CT is associated with higher yield, biomass and generally a higher performing genotype. In view of changing climatic conditions, measuring CT is becoming increasingly important in breeding and variety testing. Ideally, CTs should be measured as simultaneously as possible in all genotypes to avoid any bias resulting from changes in environmental conditions. The use of thermal cameras mounted on drones allows to measure large experiments in a short time. Uncooled thermal cameras are sufficiently lightweight to be mounted on drones. However, such cameras are prone to thermal drift, where the measured temperature changes with the conditions the sensor is exposed to. Thermal drift and changing environmental conditions impede precise and consistent thermal measurements with uncooled cameras. Furthermore, the viewing geometry of images affects the ratio between pixels showing soil or plants. Particularly for row crops such as wheat, changing viewing geometries will increase CT uncertainties. Restricting the range of viewing geometries can potentially reduce these effects. In this study, sequences of repeated thermal images were analyzed in a multi-view approach which allowed to extract information on trigger timing and viewing geometry for individual measurements. We propose a mixed model approach that can account for temporal drift and viewing geometry by including temporal and geometric covariates. This approach allowed to improve consistency and genotype specificity of CT measurements compared to approaches relying on orthomosaics in a two-year field variety testing trial with winter wheat. The correlations between independent measurements taken within 20 min reached 0.99, and heritabilities 0.95. Selecting measurements with oblique viewing geometries for analysis can reduce the influence of soil background. The proposed workflow provides a lean phenotyping method to collect high-quality CT measurements in terms of ranking consistency and heritability with an affordable thermal camera by incorporating available additional information from drone-based mapping flights in a post-processing step.
冠层温度(CT)是一种综合性状,表明植物基因型对环境的相对适应性。CT越低,产量和生物量越高,基因型的表现也越好。鉴于气候条件不断变化,测量 CT 在育种和品种测试中变得越来越重要。理想情况下,应尽可能同时测量所有基因型的 CT,以避免因环境条件变化而产生偏差。使用安装在无人机上的红外热像仪可以在短时间内测量大量实验数据。非制冷型热像仪非常轻便,可以安装在无人机上。不过,这种热像仪容易发生热漂移,即测量温度会随着传感器所处环境的变化而变化。热漂移和不断变化的环境条件阻碍了使用非制冷型热像仪进行精确一致的热测量。此外,图像的观察几何形状会影响显示土壤或植物的像素之间的比例。特别是对于小麦等行列作物,不断变化的观察几何形状会增加 CT 的不确定性。限制观察几何形状的范围有可能减少这些影响。在这项研究中,我们采用多视角方法对重复热图像序列进行了分析,从而提取出单个测量的触发时机和观察几何的信息。我们提出了一种混合模型方法,通过加入时间和几何协变量来解释时间漂移和观察几何。在一项为期两年的冬小麦田间品种测试试验中,与依赖正交马赛克的方法相比,这种方法提高了 CT 测量的一致性和基因型特异性。20 分钟内进行的独立测量之间的相关性达到 0.99,遗传率达到 0.95。选择斜视几何图形进行测量分析可以减少土壤背景的影响。建议的工作流程提供了一种简便的表型方法,通过在后处理步骤中纳入无人机测绘飞行的可用附加信息,利用经济实惠的热像仪收集高质量的 CT 测量结果,从而获得等级一致性和遗传力。
{"title":"Improving drone-based uncalibrated estimates of wheat canopy temperature in plot experiments by accounting for confounding factors in a multi-view analysis","authors":"Simon Treier ,&nbsp;Juan M. Herrera ,&nbsp;Andreas Hund ,&nbsp;Norbert Kirchgessner ,&nbsp;Helge Aasen ,&nbsp;Achim Walter ,&nbsp;Lukas Roth","doi":"10.1016/j.isprsjprs.2024.09.015","DOIUrl":"10.1016/j.isprsjprs.2024.09.015","url":null,"abstract":"<div><div>Canopy temperature (CT) is an integrative trait, indicative of the relative fitness of a plant genotype to the environment. Lower CT is associated with higher yield, biomass and generally a higher performing genotype. In view of changing climatic conditions, measuring CT is becoming increasingly important in breeding and variety testing. Ideally, CTs should be measured as simultaneously as possible in all genotypes to avoid any bias resulting from changes in environmental conditions. The use of thermal cameras mounted on drones allows to measure large experiments in a short time. Uncooled thermal cameras are sufficiently lightweight to be mounted on drones. However, such cameras are prone to thermal drift, where the measured temperature changes with the conditions the sensor is exposed to. Thermal drift and changing environmental conditions impede precise and consistent thermal measurements with uncooled cameras. Furthermore, the viewing geometry of images affects the ratio between pixels showing soil or plants. Particularly for row crops such as wheat, changing viewing geometries will increase CT uncertainties. Restricting the range of viewing geometries can potentially reduce these effects. In this study, sequences of repeated thermal images were analyzed in a multi-view approach which allowed to extract information on trigger timing and viewing geometry for individual measurements. We propose a mixed model approach that can account for temporal drift and viewing geometry by including temporal and geometric covariates. This approach allowed to improve consistency and genotype specificity of CT measurements compared to approaches relying on orthomosaics in a two-year field variety testing trial with winter wheat. The correlations between independent measurements taken within 20<!--> <!-->min reached 0.99, and heritabilities 0.95. Selecting measurements with oblique viewing geometries for analysis can reduce the influence of soil background. The proposed workflow provides a lean phenotyping method to collect high-quality CT measurements in terms of ranking consistency and heritability with an affordable thermal camera by incorporating available additional information from drone-based mapping flights in a post-processing step.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 721-741"},"PeriodicalIF":10.6,"publicationDate":"2024-10-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426555","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
SAMPolyBuild: Adapting the Segment Anything Model for polygonal building extraction SAMPolyBuild:为多边形建筑提取调整 "分段任意模型
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-10 DOI: 10.1016/j.isprsjprs.2024.09.018
Chenhao Wang , Jingbo Chen , Yu Meng , Yupeng Deng , Kai Li , Yunlong Kong
Extracting polygonal buildings from high-resolution remote sensing images is a critical task for large-scale mapping, 3D city modeling, and various geographic information system applications. Traditional methods are often restricted in accurately delineating boundaries and exhibit limited generalizability, which can affect their real-world applicability. The Segment Anything Model (SAM), a promptable segmentation model trained on an unprecedentedly large dataset, demonstrates remarkable generalization ability across various scenarios. In this context, we present SAMPolyBuild, an innovative framework that adapts SAM for polygonal building extraction, allowing for both automatic and prompt-based extraction. To fulfill the requirement for object location prompts in SAM, we developed the Auto Bbox Prompter, which is trained to detect building bounding boxes directly from the image encoder features of the SAM. The boundary precision of the SAM mask results was insufficient for vector polygon extraction, especially when challenged by blurry edges and tree occlusions. Therefore, we extended the SAM decoder with additional parameters to enable multitask learning to predict masks and generate Gaussian vertex and boundary maps simultaneously. Furthermore, we developed a mask-guided vertex connection algorithm to generate the final polygon. Extensive evaluation on the WHU-Mix vector dataset and SpaceNet datasets demonstrate that our method achieves a new state-of-the-art in terms of accuracy and generalizability, significantly improving average precision (AP), average recall (AR), intersection over union (IoU), boundary F1, and vertex F1 metrics. Moreover, by combining the automatic and prompt modes of our framework, we found that 91.2% of the building polygons predicted by SAMPolyBuild on out-of-domain data closely match the quality of manually delineated polygons. The source code is available at https://github.com/wchh-2000/SAMPolyBuild.
从高分辨率遥感图像中提取多边形建筑物是大比例尺测绘、三维城市建模和各种地理信息系统应用的一项关键任务。传统方法在准确划定边界方面往往受到限制,并表现出有限的通用性,这可能会影响其在现实世界中的适用性。Segment Anything Model(SAM)是一种在前所未有的大型数据集上训练出来的可提示分割模型,它在各种场景中都表现出卓越的泛化能力。在这种情况下,我们提出了 SAMPolyBuild,这是一个创新的框架,它将 SAM 用于多边形建筑提取,允许自动提取和基于提示的提取。为了满足 SAM 中对象位置提示的要求,我们开发了自动边界框提示器,经过训练,它可以直接从 SAM 的图像编码器特征中检测建筑物边界框。SAM 掩码结果的边界精度不足以进行矢量多边形提取,尤其是在边缘模糊和树木遮挡的情况下。因此,我们对 SAM 解码器进行了扩展,增加了额外的参数,使多任务学习能够预测遮罩,并同时生成高斯顶点和边界图。此外,我们还开发了一种掩膜引导的顶点连接算法,以生成最终的多边形。在WHU-Mix向量数据集和SpaceNet数据集上进行的广泛评估表明,我们的方法在准确性和通用性方面达到了新的先进水平,显著提高了平均精确度(AP)、平均召回率(AR)、交集大于联合(IoU)、边界F1和顶点F1指标。此外,通过结合我们框架的自动和提示模式,我们发现 SAMPolyBuild 在域外数据上预测的 91.2% 的建筑多边形与人工划定的多边形质量非常接近。源代码见 https://github.com/wchh-2000/SAMPolyBuild。
{"title":"SAMPolyBuild: Adapting the Segment Anything Model for polygonal building extraction","authors":"Chenhao Wang ,&nbsp;Jingbo Chen ,&nbsp;Yu Meng ,&nbsp;Yupeng Deng ,&nbsp;Kai Li ,&nbsp;Yunlong Kong","doi":"10.1016/j.isprsjprs.2024.09.018","DOIUrl":"10.1016/j.isprsjprs.2024.09.018","url":null,"abstract":"<div><div>Extracting polygonal buildings from high-resolution remote sensing images is a critical task for large-scale mapping, 3D city modeling, and various geographic information system applications. Traditional methods are often restricted in accurately delineating boundaries and exhibit limited generalizability, which can affect their real-world applicability. The Segment Anything Model (SAM), a promptable segmentation model trained on an unprecedentedly large dataset, demonstrates remarkable generalization ability across various scenarios. In this context, we present SAMPolyBuild, an innovative framework that adapts SAM for polygonal building extraction, allowing for both automatic and prompt-based extraction. To fulfill the requirement for object location prompts in SAM, we developed the Auto Bbox Prompter, which is trained to detect building bounding boxes directly from the image encoder features of the SAM. The boundary precision of the SAM mask results was insufficient for vector polygon extraction, especially when challenged by blurry edges and tree occlusions. Therefore, we extended the SAM decoder with additional parameters to enable multitask learning to predict masks and generate Gaussian vertex and boundary maps simultaneously. Furthermore, we developed a mask-guided vertex connection algorithm to generate the final polygon. Extensive evaluation on the WHU-Mix vector dataset and SpaceNet datasets demonstrate that our method achieves a new state-of-the-art in terms of accuracy and generalizability, significantly improving average precision (AP), average recall (AR), intersection over union (IoU), boundary F1, and vertex F1 metrics. Moreover, by combining the automatic and prompt modes of our framework, we found that 91.2% of the building polygons predicted by SAMPolyBuild on out-of-domain data closely match the quality of manually delineated polygons. The source code is available at <span><span>https://github.com/wchh-2000/SAMPolyBuild</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 707-720"},"PeriodicalIF":10.6,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426649","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
PolyGNN: Polyhedron-based graph neural network for 3D building reconstruction from point clouds PolyGNN:基于多面体的图神经网络,用于从点云重建 3D 建筑
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-10 DOI: 10.1016/j.isprsjprs.2024.09.031
Zhaiyu Chen , Yilei Shi , Liangliang Nan , Zhitong Xiong , Xiao Xiang Zhu
We present PolyGNN, a polyhedron-based graph neural network for 3D building reconstruction from point clouds. PolyGNN learns to assemble primitives obtained by polyhedral decomposition via graph node classification, achieving a watertight and compact reconstruction. To effectively represent arbitrary-shaped polyhedra in the neural network, we propose a skeleton-based sampling strategy to generate polyhedron-wise queries. These queries are then incorporated with inter-polyhedron adjacency to enhance the classification. PolyGNN is end-to-end optimizable and is designed to accommodate variable-size input points, polyhedra, and queries with an index-driven batching technique. To address the abstraction gap between existing city-building models and the underlying instances, and provide a fair evaluation of the proposed method, we develop our method on a large-scale synthetic dataset with well-defined ground truths of polyhedral labels. We further conduct a transferability analysis across cities and on real-world point clouds. Both qualitative and quantitative results demonstrate the effectiveness of our method, particularly its efficiency for large-scale reconstructions. The source code and data are available at https://github.com/chenzhaiyu/polygnn.
我们介绍的 PolyGNN 是一种基于多面体的图神经网络,用于从点云重建三维建筑物。PolyGNN 通过图节点分类来学习组装多面体分解获得的基元,从而实现无缝、紧凑的重建。为了在神经网络中有效地表示任意形状的多面体,我们提出了一种基于骨架的采样策略来生成多面体查询。然后将这些查询与多面体间的邻接关系结合起来,以增强分类效果。PolyGNN 可进行端到端优化,并采用索引驱动的批处理技术,可容纳不同大小的输入点、多面体和查询。为了解决现有城市建设模型与底层实例之间的抽象差距,并对所提出的方法进行公平评估,我们在一个大规模合成数据集上开发了我们的方法,该数据集具有定义明确的多面体标签地面真相。我们还进一步进行了跨城市和真实世界点云的可移植性分析。定性和定量结果都证明了我们方法的有效性,尤其是它在大规模重建中的效率。源代码和数据可在 https://github.com/chenzhaiyu/polygnn 上获取。
{"title":"PolyGNN: Polyhedron-based graph neural network for 3D building reconstruction from point clouds","authors":"Zhaiyu Chen ,&nbsp;Yilei Shi ,&nbsp;Liangliang Nan ,&nbsp;Zhitong Xiong ,&nbsp;Xiao Xiang Zhu","doi":"10.1016/j.isprsjprs.2024.09.031","DOIUrl":"10.1016/j.isprsjprs.2024.09.031","url":null,"abstract":"<div><div>We present PolyGNN, a polyhedron-based graph neural network for 3D building reconstruction from point clouds. PolyGNN learns to assemble primitives obtained by polyhedral decomposition via graph node classification, achieving a watertight and compact reconstruction. To effectively represent arbitrary-shaped polyhedra in the neural network, we propose a skeleton-based sampling strategy to generate polyhedron-wise queries. These queries are then incorporated with inter-polyhedron adjacency to enhance the classification. PolyGNN is end-to-end optimizable and is designed to accommodate variable-size input points, polyhedra, and queries with an index-driven batching technique. To address the abstraction gap between existing city-building models and the underlying instances, and provide a fair evaluation of the proposed method, we develop our method on a large-scale synthetic dataset with well-defined ground truths of polyhedral labels. We further conduct a transferability analysis across cities and on real-world point clouds. Both qualitative and quantitative results demonstrate the effectiveness of our method, particularly its efficiency for large-scale reconstructions. The source code and data are available at <span><span>https://github.com/chenzhaiyu/polygnn</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 693-706"},"PeriodicalIF":10.6,"publicationDate":"2024-10-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426646","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Clustering, triangulation, and evaluation of 3D lines in multiple images 多幅图像中三维线条的聚类、三角测量和评估
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-08 DOI: 10.1016/j.isprsjprs.2024.10.001
Dong Wei , Haoyu Guo , Yi Wan , Yongjun Zhang , Chang Li , Guangshuai Wang
Three-dimensional (3D) lines require further enhancement in both clustering and triangulation. Line clustering assigns multiple image lines to a single 3D line to eliminate redundant 3D lines. Currently, it depends on the fixed and empirical parameter. However, a loose parameter could lead to over-clustering, while a strict one may cause redundant 3D lines. Due to the absence of the ground truth, the assessment of line clustering remains unexplored. Additionally, 3D line triangulation, which determines the 3D line segment in object space, is prone to failure due to its sensitivity to positional and camera errors.
This paper aims to improve the clustering and triangulation of 3D lines and to offer a reliable evaluation method. (1) To achieve accurate clustering, we introduce a probability model, which uses the prior error of the structure from the motion, to determine adaptive thresholds; thus controlling false clustering caused by the fixed hyperparameter. (2) For robust triangulation, we employ a universal framework that refines the 3D line with various forms of geometric consistency. (3) For a reliable evaluation, we investigate consistent patterns in urban environments to evaluate the clustering and triangulation, eliminating the need to manually draw the ground truth.
To evaluate our method, we utilized datasets of Internet image, totaling over ten thousand images, alongside aerial images with dimensions exceeding ten thousand pixels. We compared our approach to state-of-the-art methods, including Line3D++, Limap, and ELSR. In these datasets, our method demonstrated improvements in clustering and triangulation accuracy by at least 20% and 3%, respectively. Additionally, our method ranked second in execution speed, surpassed only by ELSR, the current fastest algorithm. The C++ source code for the proposed algorithm, along with the dataset used in this paper, is available at https://github.com/weidong-whu/3DLineResconstruction.
三维(3D)线条需要在聚类和三角测量方面进一步改进。线条聚类将多条图像线条分配给一条三维线条,以消除多余的三维线条。目前,它取决于固定的经验参数。然而,松散的参数可能导致过度聚类,而严格的参数则可能造成冗余三维线。由于缺乏地面实况,对线条聚类的评估仍有待探索。此外,三维线三角测量确定了物体空间中的三维线段,但由于其对位置误差和相机误差的敏感性而容易失败。(1) 为了实现精确聚类,我们引入了一个概率模型,利用运动结构的先验误差来确定自适应阈值,从而控制由固定超参数引起的错误聚类。(2) 为了实现稳健的三角测量,我们采用了一个通用框架,通过各种形式的几何一致性来完善三维线。(3) 为了进行可靠的评估,我们调查了城市环境中的一致模式,以评估聚类和三角测量,从而消除了手动绘制地面实况的需要。为了评估我们的方法,我们使用了互联网图像数据集,总计超过 1 万张图像,以及尺寸超过 1 万像素的航空图像。我们将我们的方法与最先进的方法进行了比较,包括 Line3D++、Limap 和 ELSR。在这些数据集中,我们的方法在聚类和三角测量精度方面分别提高了至少 20% 和 3%。此外,我们的方法在执行速度上排名第二,仅次于目前最快的算法 ELSR。本文中使用的数据集和算法的 C++ 源代码可在 https://github.com/weidong-whu/3DLineResconstruction 上获取。
{"title":"Clustering, triangulation, and evaluation of 3D lines in multiple images","authors":"Dong Wei ,&nbsp;Haoyu Guo ,&nbsp;Yi Wan ,&nbsp;Yongjun Zhang ,&nbsp;Chang Li ,&nbsp;Guangshuai Wang","doi":"10.1016/j.isprsjprs.2024.10.001","DOIUrl":"10.1016/j.isprsjprs.2024.10.001","url":null,"abstract":"<div><div>Three-dimensional (3D) lines require further enhancement in both clustering and triangulation. Line clustering assigns multiple image lines to a single 3D line to eliminate redundant 3D lines. Currently, it depends on the fixed and empirical parameter. However, a loose parameter could lead to over-clustering, while a strict one may cause redundant 3D lines. Due to the absence of the ground truth, the assessment of line clustering remains unexplored. Additionally, 3D line triangulation, which determines the 3D line segment in object space, is prone to failure due to its sensitivity to positional and camera errors.</div><div>This paper aims to improve the clustering and triangulation of 3D lines and to offer a reliable evaluation method. (1) To achieve accurate clustering, we introduce a probability model, which uses the prior error of the structure from the motion, to determine adaptive thresholds; thus controlling false clustering caused by the fixed hyperparameter. (2) For robust triangulation, we employ a universal framework that refines the 3D line with various forms of geometric consistency. (3) For a reliable evaluation, we investigate consistent patterns in urban environments to evaluate the clustering and triangulation, eliminating the need to manually draw the ground truth.</div><div>To evaluate our method, we utilized datasets of Internet image, totaling over ten thousand images, alongside aerial images with dimensions exceeding ten thousand pixels. We compared our approach to state-of-the-art methods, including Line3D++, Limap, and ELSR. In these datasets, our method demonstrated improvements in clustering and triangulation accuracy by at least 20% and 3%, respectively. Additionally, our method ranked second in execution speed, surpassed only by ELSR, the current fastest algorithm. The C++ source code for the proposed algorithm, along with the dataset used in this paper, is available at <span><span>https://github.com/weidong-whu/3DLineResconstruction</span><svg><path></path></svg></span>.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 678-692"},"PeriodicalIF":10.6,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426645","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
HyperDehazing: A hyperspectral image dehazing benchmark dataset and a deep learning model for haze removal HyperDehazing:高光谱图像去雾基准数据集和用于去除雾霾的深度学习模型
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-05 DOI: 10.1016/j.isprsjprs.2024.09.034
Hang Fu , Ziyan Ling , Genyun Sun , Jinchang Ren , Aizhu Zhang , Li Zhang , Xiuping Jia
Haze contamination severely degrades the quality and accuracy of optical remote sensing (RS) images, including hyperspectral images (HSIs). Currently, there are no paired benchmark datasets containing hazy and haze-free scenes in HSI dehazing, and few studies have analyzed the distributional properties of haze in the spatial and spectral domains. In this paper, we developed a new hazy synthesis strategy and constructed the first hyperspectral dehazing benchmark dataset (HyperDehazing), which contains 2000 pairs synthetic HSIs covering 100 scenes and another 70 real hazy HSIs. By analyzing the distribution characteristics of haze, we further proposed a deep learning model called HyperDehazeNet for haze removal from HSIs. Haze-insensitive longwave information injection, novel attention mechanisms, spectral loss function, and residual learning are used to improve dehazing and scene reconstruction capability. Comprehensive experimental results demonstrate that the HyperDehazing dataset effectively represents complex haze in real scenes with synthetic authenticity and scene diversity, establishing itself as a new benchmark for training and assessment of HSI dehazing methods. Experimental results on the HyperDehazing dataset demonstrate that our proposed HyperDehazeNet effectively removes complex haze from HSIs, with outstanding spectral reconstruction and feature differentiation capabilities. Furthermore, additional experiments conducted on real HSIs as well as the widely used Landsat-8 and Sentinel-2 datasets showcase the exceptional dehazing performance and robust generalization capabilities of HyperDehazeNet. Our method surpasses other state-of-the-art methods with high computational efficiency and a low number of parameters.
雾霾污染严重降低了光学遥感(RS)图像(包括高光谱图像)的质量和精度。目前,在高光谱图像去噪中还没有包含雾霾和无雾霾场景的配对基准数据集,也很少有研究分析雾霾在空间和光谱领域的分布特性。在本文中,我们开发了一种新的雾度合成策略,并构建了第一个高光谱去雾基准数据集(HyperDehazing),其中包含 2000 对合成 HSI(覆盖 100 个场景)和另外 70 个真实雾度 HSI。通过分析雾霾的分布特征,我们进一步提出了一种名为HyperDehazeNet的深度学习模型,用于从高光谱图像中去除雾霾。该模型采用了对雾霾不敏感的长波信息注入、新颖的注意力机制、光谱损失函数和残差学习等方法来提高去雾霾和场景重建能力。全面的实验结果表明,HyperDehazing 数据集有效地表现了真实场景中的复杂雾霾,具有合成真实性和场景多样性,成为训练和评估 HSI 去雾霾方法的新基准。在 HyperDehazing 数据集上的实验结果表明,我们提出的 HyperDehazeNet 能有效去除 HSI 中的复杂雾度,并具有出色的光谱重建和特征区分能力。此外,在真实的 HSI 以及广泛使用的 Landsat-8 和 Sentinel-2 数据集上进行的其他实验也证明了 HyperDehazeNet 卓越的去雾性能和强大的泛化能力。我们的方法计算效率高、参数数量少,超越了其他最先进的方法。
{"title":"HyperDehazing: A hyperspectral image dehazing benchmark dataset and a deep learning model for haze removal","authors":"Hang Fu ,&nbsp;Ziyan Ling ,&nbsp;Genyun Sun ,&nbsp;Jinchang Ren ,&nbsp;Aizhu Zhang ,&nbsp;Li Zhang ,&nbsp;Xiuping Jia","doi":"10.1016/j.isprsjprs.2024.09.034","DOIUrl":"10.1016/j.isprsjprs.2024.09.034","url":null,"abstract":"<div><div>Haze contamination severely degrades the quality and accuracy of optical remote sensing (RS) images, including hyperspectral images (HSIs). Currently, there are no paired benchmark datasets containing hazy and haze-free scenes in HSI dehazing, and few studies have analyzed the distributional properties of haze in the spatial and spectral domains. In this paper, we developed a new hazy synthesis strategy and constructed the first hyperspectral dehazing benchmark dataset (HyperDehazing), which contains 2000 pairs synthetic HSIs covering 100 scenes and another 70 real hazy HSIs. By analyzing the distribution characteristics of haze, we further proposed a deep learning model called HyperDehazeNet for haze removal from HSIs. Haze-insensitive longwave information injection, novel attention mechanisms, spectral loss function, and residual learning are used to improve dehazing and scene reconstruction capability. Comprehensive experimental results demonstrate that the HyperDehazing dataset effectively represents complex haze in real scenes with synthetic authenticity and scene diversity, establishing itself as a new benchmark for training and assessment of HSI dehazing methods. Experimental results on the HyperDehazing dataset demonstrate that our proposed HyperDehazeNet effectively removes complex haze from HSIs, with outstanding spectral reconstruction and feature differentiation capabilities. Furthermore, additional experiments conducted on real HSIs as well as the widely used Landsat-8 and Sentinel-2 datasets showcase the exceptional dehazing performance and robust generalization capabilities of HyperDehazeNet. Our method surpasses other state-of-the-art methods with high computational efficiency and a low number of parameters.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 663-677"},"PeriodicalIF":10.6,"publicationDate":"2024-10-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426644","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
Vision-guided robot calibration using photogrammetric methods 使用摄影测量方法进行视觉引导机器人校准
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-04 DOI: 10.1016/j.isprsjprs.2024.09.037
Markus Ulrich , Carsten Steger , Florian Butsch , Maurice Liebe
We propose novel photogrammetry-based robot calibration methods for industrial robots that are guided by cameras or 3D sensors. Compared to state-of-the-art methods, our methods are capable of calibrating the robot kinematics, the hand–eye transformations, and, for camera-guided robots, the interior orientation of the camera simultaneously. Our approach uses a minimal parameterization of the robot kinematics and hand–eye transformations. Furthermore, it uses a camera model that is capable of handling a large range of complex lens distortions that can occur in cameras that are typically used in machine vision applications. To determine the model parameters, geometrically meaningful photogrammetric error measures are used. They are independent of the parameterization of the model and typically result in a higher accuracy. We apply a stochastic model for all parameters (observations and unknowns), which allows us to assess the precision and significance of the calibrated model parameters. To evaluate our methods, we propose novel procedures that are relevant in real-world applications and do not require ground truth values. Experiments on synthetic and real data show that our approach improves the absolute positioning accuracy of industrial robots significantly. By applying our approach to two different uncalibrated UR3e robots, one guided by a camera and one by a 3D sensor, we were able to reduce the RMS evaluation error by approximately 85% for each robot.
我们针对由摄像头或三维传感器引导的工业机器人提出了基于摄影测量的新型机器人校准方法。与最先进的方法相比,我们的方法能够同时校准机器人运动学、手眼变换,对于摄像头引导的机器人,还能同时校准摄像头的内部方位。我们的方法使用了机器人运动学和手眼变换的最小参数化。此外,它使用的相机模型能够处理机器视觉应用中通常使用的相机中可能出现的大量复杂镜头变形。为了确定模型参数,使用了具有几何意义的摄影测量误差测量方法。它们与模型的参数化无关,通常能获得更高的精度。我们对所有参数(观测值和未知数)采用随机模型,这样就能评估校准模型参数的精度和重要性。为了评估我们的方法,我们提出了与实际应用相关的新程序,这些程序不需要地面真实值。合成数据和真实数据的实验表明,我们的方法显著提高了工业机器人的绝对定位精度。通过将我们的方法应用于两个不同的未校准 UR3e 机器人(一个由摄像头引导,一个由 3D 传感器引导),我们能够将每个机器人的均方根评估误差降低约 85%。
{"title":"Vision-guided robot calibration using photogrammetric methods","authors":"Markus Ulrich ,&nbsp;Carsten Steger ,&nbsp;Florian Butsch ,&nbsp;Maurice Liebe","doi":"10.1016/j.isprsjprs.2024.09.037","DOIUrl":"10.1016/j.isprsjprs.2024.09.037","url":null,"abstract":"<div><div>We propose novel photogrammetry-based robot calibration methods for industrial robots that are guided by cameras or 3D sensors. Compared to state-of-the-art methods, our methods are capable of calibrating the robot kinematics, the hand–eye transformations, and, for camera-guided robots, the interior orientation of the camera simultaneously. Our approach uses a minimal parameterization of the robot kinematics and hand–eye transformations. Furthermore, it uses a camera model that is capable of handling a large range of complex lens distortions that can occur in cameras that are typically used in machine vision applications. To determine the model parameters, geometrically meaningful photogrammetric error measures are used. They are independent of the parameterization of the model and typically result in a higher accuracy. We apply a stochastic model for all parameters (observations and unknowns), which allows us to assess the precision and significance of the calibrated model parameters. To evaluate our methods, we propose novel procedures that are relevant in real-world applications and do not require ground truth values. Experiments on synthetic and real data show that our approach improves the absolute positioning accuracy of industrial robots significantly. By applying our approach to two different uncalibrated UR3e robots, one guided by a camera and one by a 3D sensor, we were able to reduce the RMS evaluation error by approximately 85% for each robot.</div></div>","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 645-662"},"PeriodicalIF":10.6,"publicationDate":"2024-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426560","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
A phenological-knowledge-independent method for automatic paddy rice mapping with time series of polarimetric SAR images 利用偏振合成孔径雷达图像时间序列进行水稻自动测绘的不依赖物候知识的方法
IF 10.6 1区 地球科学 Q1 GEOGRAPHY, PHYSICAL Pub Date : 2024-10-04 DOI: 10.1016/j.isprsjprs.2024.09.035
Suya Lin , Zhixin Qi , Xia Li , Hui Zhang , Qianwen Lv , Di Huang
<div><div>Paddy rice, which sustains more than half of the global population, requires accurate and efficient mapping to ensure food security. Synthetic aperture radar (SAR) has become indispensable in this process due to its remarkable ability to operate effectively in adverse weather conditions and its sensitivity to paddy rice growth. Phenological-knowledge-based (PKB) methods have been commonly employed in conjunction with time series of SAR images for paddy rice mapping, primarily because they eliminate the need for training datasets. However, PKB methods possess inherent limitations, primarily stemming from their reliance on precise phenological information regarding paddy rice growth. This information varies across regions and paddy rice varieties, making it challenging to use PKB methods effectively on a large spatial scale, such as the national or global scale, where collecting comprehensive phenological data becomes impractical. Moreover, variations in farming practices and field conditions can lead to differences in paddy rice growth stages even within the same region. Using a generalized set of phenological knowledge in PKB methods may not be suitable for all paddy fields, potentially resulting in errors in paddy rice extraction. To address the challenges posed by PKB methods, this study proposed an innovative approach known as the phenological-knowledge-independent (PKI) method for mapping paddy rice using time series of Sentinel-1 SAR images. The central innovation of the PKI method lies in its capability to map paddy rice without relying on specific knowledge of paddy rice phenology or the need for a training dataset. This was made possible by the incorporation of three novel metrics: VH and VV normalized maximum temporal changes (NMTC) and VH temporal mean, derived from the distinctions between paddy rice and other land cover types in time series of SAR images. The PKI method was rigorously evaluated across three regions in China, each featuring different paddy rice varieties. Additionally, the PKI method was compared with two prevalent phenological-knowledge-based techniques: the automated paddy rice mapping method using SAR flooding signals (ARM-SARFS) and the manual interpretation of unsupervised clustering results (MI-UCR). The PKI method achieved an average overall accuracy of 97.99%, surpassing the ARM-SARFS, which recorded an accuracy of 89.65% due to errors stemming from phenological disparities among different paddy fields. Furthermore, the PKI method delivered results on par with the MI-UCR, which relied on the fusion of SAR and optical image time series, achieving an accuracy of 97.71%. As demonstrated by these findings, the PKI method proves highly effective in mapping paddy rice across diverse regions, all without the need for phenological knowledge or a training dataset. Consequently, it holds substantial promise for efficiently mapping paddy rice on a large spatial scale. The source code used in this study is availa
水稻养活了全球一半以上的人口,因此需要精确高效的测绘来确保粮食安全。合成孔径雷达(SAR)因其在恶劣天气条件下有效工作的卓越能力和对水稻生长的敏感性,已成为这一过程中不可或缺的工具。基于物候知识(PKB)的方法通常与合成孔径雷达图像时间序列结合使用,用于水稻测绘,这主要是因为这些方法无需训练数据集。然而,PKB 方法有其固有的局限性,主要是因为它们依赖于有关水稻生长的精确物候信息。这些信息因地区和水稻品种的不同而各异,因此在大空间尺度(如全国或全球尺度)上有效使用 PKB 方法具有挑战性,因为在这些尺度上收集全面的物候数据是不切实际的。此外,即使在同一地区,耕作方式和田间条件的不同也会导致水稻生长阶段的差异。在 PKB 方法中使用一套通用的物候知识可能并不适用于所有稻田,从而可能导致水稻提取错误。针对 PKB 方法带来的挑战,本研究提出了一种创新方法,即独立于物候知识(PKI)方法,用于利用 Sentinel-1 SAR 图像的时间序列绘制水稻图。PKI 方法的核心创新点在于它无需依赖特定的水稻物候知识或训练数据集即可绘制水稻图。这要归功于三个新指标:VH 和 VV 归一化最大时间变化 (NMTC) 以及 VH 时间平均值,这三个指标是从 SAR 图像时间序列中水稻与其他土地覆被类型的区别中得出的。PKI 方法在中国三个地区进行了严格评估,每个地区都有不同的水稻品种。此外,PKI 方法还与两种常用的基于物候知识的技术进行了比较:利用 SAR 水浸信号的水稻自动绘图方法(ARM-SARFS)和对无监督聚类结果的人工解释(MI-UCR)。PKI 方法的平均总体准确率达到 97.99%,超过了 ARM-SARFS,后者的准确率为 89.65%,原因是不同稻田之间的物候差异造成了误差。此外,PKI 方法与 MI-UCR 方法的结果相当,后者依赖于合成孔径雷达和光学图像时间序列的融合,准确率达到 97.71%。这些研究结果表明,PKI 方法在绘制不同地区的水稻地图时非常有效,而且无需物候知识或训练数据集。因此,该方法有望在大空间尺度上高效绘制水稻图谱。本研究使用的源代码可在 https://code.earthengine.google.com/f82cf10cad64fa3f971ae99027001a6e 上获取。
{"title":"A phenological-knowledge-independent method for automatic paddy rice mapping with time series of polarimetric SAR images","authors":"Suya Lin ,&nbsp;Zhixin Qi ,&nbsp;Xia Li ,&nbsp;Hui Zhang ,&nbsp;Qianwen Lv ,&nbsp;Di Huang","doi":"10.1016/j.isprsjprs.2024.09.035","DOIUrl":"10.1016/j.isprsjprs.2024.09.035","url":null,"abstract":"&lt;div&gt;&lt;div&gt;Paddy rice, which sustains more than half of the global population, requires accurate and efficient mapping to ensure food security. Synthetic aperture radar (SAR) has become indispensable in this process due to its remarkable ability to operate effectively in adverse weather conditions and its sensitivity to paddy rice growth. Phenological-knowledge-based (PKB) methods have been commonly employed in conjunction with time series of SAR images for paddy rice mapping, primarily because they eliminate the need for training datasets. However, PKB methods possess inherent limitations, primarily stemming from their reliance on precise phenological information regarding paddy rice growth. This information varies across regions and paddy rice varieties, making it challenging to use PKB methods effectively on a large spatial scale, such as the national or global scale, where collecting comprehensive phenological data becomes impractical. Moreover, variations in farming practices and field conditions can lead to differences in paddy rice growth stages even within the same region. Using a generalized set of phenological knowledge in PKB methods may not be suitable for all paddy fields, potentially resulting in errors in paddy rice extraction. To address the challenges posed by PKB methods, this study proposed an innovative approach known as the phenological-knowledge-independent (PKI) method for mapping paddy rice using time series of Sentinel-1 SAR images. The central innovation of the PKI method lies in its capability to map paddy rice without relying on specific knowledge of paddy rice phenology or the need for a training dataset. This was made possible by the incorporation of three novel metrics: VH and VV normalized maximum temporal changes (NMTC) and VH temporal mean, derived from the distinctions between paddy rice and other land cover types in time series of SAR images. The PKI method was rigorously evaluated across three regions in China, each featuring different paddy rice varieties. Additionally, the PKI method was compared with two prevalent phenological-knowledge-based techniques: the automated paddy rice mapping method using SAR flooding signals (ARM-SARFS) and the manual interpretation of unsupervised clustering results (MI-UCR). The PKI method achieved an average overall accuracy of 97.99%, surpassing the ARM-SARFS, which recorded an accuracy of 89.65% due to errors stemming from phenological disparities among different paddy fields. Furthermore, the PKI method delivered results on par with the MI-UCR, which relied on the fusion of SAR and optical image time series, achieving an accuracy of 97.71%. As demonstrated by these findings, the PKI method proves highly effective in mapping paddy rice across diverse regions, all without the need for phenological knowledge or a training dataset. Consequently, it holds substantial promise for efficiently mapping paddy rice on a large spatial scale. The source code used in this study is availa","PeriodicalId":50269,"journal":{"name":"ISPRS Journal of Photogrammetry and Remote Sensing","volume":"218 ","pages":"Pages 628-644"},"PeriodicalIF":10.6,"publicationDate":"2024-10-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142426561","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"地球科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
引用次数: 0
期刊
ISPRS Journal of Photogrammetry and Remote Sensing
全部 Acc. Chem. Res. ACS Applied Bio Materials ACS Appl. Electron. Mater. ACS Appl. Energy Mater. ACS Appl. Mater. Interfaces ACS Appl. Nano Mater. ACS Appl. Polym. Mater. ACS BIOMATER-SCI ENG ACS Catal. ACS Cent. Sci. ACS Chem. Biol. ACS Chemical Health & Safety ACS Chem. Neurosci. ACS Comb. Sci. ACS Earth Space Chem. ACS Energy Lett. ACS Infect. Dis. ACS Macro Lett. ACS Mater. Lett. ACS Med. Chem. Lett. ACS Nano ACS Omega ACS Photonics ACS Sens. ACS Sustainable Chem. Eng. ACS Synth. Biol. Anal. Chem. BIOCHEMISTRY-US Bioconjugate Chem. BIOMACROMOLECULES Chem. Res. Toxicol. Chem. Rev. Chem. Mater. CRYST GROWTH DES ENERG FUEL Environ. Sci. Technol. Environ. Sci. Technol. Lett. Eur. J. Inorg. Chem. IND ENG CHEM RES Inorg. Chem. J. Agric. Food. Chem. J. Chem. Eng. Data J. Chem. Educ. J. Chem. Inf. Model. J. Chem. Theory Comput. J. Med. Chem. J. Nat. Prod. J PROTEOME RES J. Am. Chem. Soc. LANGMUIR MACROMOLECULES Mol. Pharmaceutics Nano Lett. Org. Lett. ORG PROCESS RES DEV ORGANOMETALLICS J. Org. Chem. J. Phys. Chem. J. Phys. Chem. A J. Phys. Chem. B J. Phys. Chem. C J. Phys. Chem. Lett. Analyst Anal. Methods Biomater. Sci. Catal. Sci. Technol. Chem. Commun. Chem. Soc. Rev. CHEM EDUC RES PRACT CRYSTENGCOMM Dalton Trans. Energy Environ. Sci. ENVIRON SCI-NANO ENVIRON SCI-PROC IMP ENVIRON SCI-WAT RES Faraday Discuss. Food Funct. Green Chem. Inorg. Chem. Front. Integr. Biol. J. Anal. At. Spectrom. J. Mater. Chem. A J. Mater. Chem. B J. Mater. Chem. C Lab Chip Mater. Chem. Front. Mater. Horiz. MEDCHEMCOMM Metallomics Mol. Biosyst. Mol. Syst. Des. Eng. Nanoscale Nanoscale Horiz. Nat. Prod. Rep. New J. Chem. Org. Biomol. Chem. Org. Chem. Front. PHOTOCH PHOTOBIO SCI PCCP Polym. Chem.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1