ISPRS Journal of Photogrammetry and Remote Sensing最新文献

LiDAR-based semantic scene understanding holds a pivotal role in various applications, including remote sensing and autonomous driving. However, the majority of LiDAR segmentation models rely on extensive and densely annotated training datasets, which is extremely laborious to annotate and hinder the widespread adoption of LiDAR systems. Semi-supervised learning (SSL) offers a promising solution by leveraging only a small amount of labeled data and a larger set of unlabeled data, aiming to train robust models with desired accuracy comparable to fully supervised learning. A typical pipeline of SSL involves the initial use of labeled data to train segmentation models, followed by the utilization of predictions generated from unlabeled data, which are used as pseudo-ground truths for model retraining. However, the scarcity of labeled data limits the capture of comprehensive representations, leading to the constraints of these pseudo-ground truths in reliability. We observed that objects captured by LiDAR sensors from varying perspectives showcase diverse data characteristics due to occlusions and distance variation, and LiDAR segmentation models trained with limited labels prove susceptible to these viewpoint disparities, resulting in inaccurately predicted pseudo-ground truths across viewpoints and the accumulation of retraining errors. To address this problem, we introduce the Temporal-Selective Guided Learning (TSG-Seg) framework. TSG-Seg explores temporal cues inherent in LiDAR frames to bridge the cross-viewpoint representations, fostering consistent and robust segmentation predictions across differing viewpoints. Specifically, we first establish point-wise correspondences across LiDAR frames with different time stamps through point registration. Subsequently, reliable point predictions are selected and propagated to points from adjacent views to the current view, serving as strong and refined supervision signals for subsequent model re-training to achieve better segmentation. We conducted extensive experiments on various SSL labeling setups across multiple public datasets, including SemanticKITTI and SemanticPOSS, to evaluate the effectiveness of TSG-Seg. Our results demonstrate its competitive performance and robustness in diverse scenarios, from data-limited to data-abundant settings. Notably, TSG-Seg achieves a mIoU of 48.6% using only 5% of and 62.3% with 40% of labeled data in the sequential split on SemanticKITTI. This consistently outperforms state-of-the-art segmentation methods, including GPC and LaserMix. These findings underscore TSG-Seg’s superior capability and potential for real-world applications. The project can be found at https://tsgseg.github.io.

The segmentation of individual trees based on deep learning is more accurate than conventional meth- ods. However, a sufficient amount of training data is mandatory to leverage the accuracy potential of deep learning-based approaches. Semi-supervised learning techniques, by contrast, can help simplify the time-consuming labelling process. In this study, we introduce a new semi-supervised tree segmen- tation approach for the precise delineation and classification of individual trees that takes advantage of pre-clustered tree training labels. Specifically, the instance segmentation Mask R-CNN is combined with the normalized cut clustering method, which is applied to lidar point clouds. The study areas were located in the Bavarian Forest National Park, southeast Germany, where the tree composition includes coniferous, deciduous and mixed forest. Important tree species are European beech (Fagus sylvatica), Norway spruce (Picea abies) and silver fir (Abies alba). Multispectral image data with a ground sample distance of 10 cm and laser scanning data with a point density of approximately 55 points/m² were acquired in June 2017. From the laser scanning data, three-channel images with a resolution of 10 cm were generated. The models were tested in seven reference plots in the national park, with a total of 516 trees measured on the ground. When the color infrared images were used, the experiments demonstrated that the Mask R-CNN models, trained with the tree labels generated through lidar-based clustering, yielded mean F1 scores of 79 % that were up to 18 % higher than those of the normalized cut baseline method and thus significantly improved. Similarly, the mean over- all accuracy of the classification results for the coniferous, deciduous, and standing deadwood tree groups was 96 % and enhanced by up to 6 % compared with the baseline classification approach. The experiments with lidar-based images yielded slightly worse (1–2 %) results both for segmentation and for classification. Our study demonstrates the utility of this simplified training data preparation pro- cedure, which leads to models trained with significantly larger amounts of data than is feasible with with manual labelling. The accuracy improvement of up to 18 % in terms of the F1 score is further evidence of its advantages.

Plant species diversity is key to ecosystem functioning, but in recent decades anthropogenic activities have prompted an alarming decline in this community trait. Thus, developing strategies to understand diversity dynamics based on affordable and efficient remote sensing monitoring is essential, as well as examining the relevance of spatial scale and vegetation structural complexity to these dynamics. Here, we used two mathematical approaches to assess the relationship between tropical woody species diversity and spectral diversity in a human-modified landscape in two vegetation types differing in their degree of complexity. Vegetation complexity was measured through the fraction of species that concentrate different proportions of the cumulative importance value index. Species diversity was assessed using Hill numbers at three spatial scales, and metrics of spectral heterogeneity, vegetation indices, as well as raw data from Landsat 9 and Sentinel-2 sensors were calculated and analysed through general linear models (GLM) and Random Forest. Vegetation complexity emerged as an important variable in modelling species from remote sensing metrics, indicating the need to model species diversity by vegetation type rather than region. Hill numbers showed different relationships with remotely sensed metrics, in consistency with the scale-dependency of ecological processes on species diversity. Contrary to multiple previous reports, in our study, GLMs produced the best fits between Hill numbers of all orders and remotely sensed metrics. If we are to meet the need of conducting efficient and speedy woody species diversity monitoring globally, we propose modelling this diversity from remotely-sensed variables as an attractive strategy, so long as the intrinsic properties of each vegetation type are acknowledged to avoid under- or overestimation biases.

Remote sensing imagery has an ever-increasing impact on important downstream applications, such as vegetation monitoring and climate change modelling. Clouds obscuring parts of the images create a substantial bottleneck in most machine learning tasks that use remote sensing data, and being robust to this issue is an important technical challenge. In many cases, cloudy images cannot be used in a machine learning pipeline, leading to either the removal of the images altogether, or to using suboptimal solutions reliant on recent cloud-free imagery or the availability of pre-trained models for the exact use case. In this work, we propose VPint2, a cloud removal method built upon the VPint algorithm, an easy-to-apply data-driven spatial interpolation method requiring no prior training, to address the problem of cloud removal. This method leverages previously sensed cloud-free images to represent the spatial structure of a region, which is then used to propagate up-to-date information from non-cloudy pixels to cloudy ones. We also created a benchmark dataset called SEN2-MSI-T, composed of 20 scenes with 5 full-sized images each, belonging to five common land cover classes. We used this dataset to evaluate our method against three alternatives: mosaicking, an AutoML-based regression method, and the nearest similar pixel interpolator. Additionally, we compared against two previously published neural network-based methods on SEN2-MSI-T, and evaluate our method on a subset of the popular SEN12MS-CR-TS benchmark dataset. The methods are compared using several performance metrics, including the structural similarity index, mean absolute error, and error rates on a downstream NDVI derivation task. Our experimental results show that VPint2 performed significantly better than competing methods over 20 experimental conditions, improving performance by 2.4% to 34.3% depending on the condition. We also found that the performance of VPint2 only decreases marginally as the temporal distance of its reference image increases, and that, unlike typical interpolation methods, the performance of VPint2 remains strong for larger percentages of cloud cover. Our findings furthermore support a cloud removal evaluation approach founded on the transfer of cloud masks over the use of cloud-free previous acquisitions as ground truth.

Floods are among the most devastating natural disasters, posing significant risks to life, property, and infrastructure globally. Earth observation satellites provide data for continuous and extensive flood monitoring, yet limitations exist in the spatial completeness of monitoring using optical images due to cloud cover. Recent studies have developed gap-filling methods for reconstructing cloud-covered areas in water maps. However, these methods are not tailored for and validated in cloudy and rainy flooding scenarios with rapid water extent changes and limited clear-sky observations, leaving room for further improvements. This study investigated and developed a novel reconstruction method for time series flood extent mapping, supporting spatially seamless monitoring of flood extents. The proposed method first identified surface water from time series images using a fine-tuned large foundation model. Then, the cloud-covered areas in the water maps were reconstructed, adhering to the introduced submaximal stability assumption, on the basis of the prior water occurrence data in the Global Surface Water dataset. The reconstructed time series water maps were refined through spatiotemporal Markov random field modeling for the final delineation of flooding areas. The effectiveness of the proposed method was evaluated with Harmonized Landsat and Sentinel-2 datasets under varying cloud cover conditions, enabling seamless flood mapping at 2–3-day frequency and 30 m resolution. Experiments at four global sites confirmed the superiority of the proposed method. It achieved higher reconstruction accuracy with average F1-scores of 0.931 during floods and 0.903 before/after floods, outperforming the typical gap-filling method with average F1-scores of 0.871 and 0.772, respectively. Additionally, the maximum flood extent maps and flood duration maps, which were composed on the basis of the reconstructed water maps, were more accurate than those using the original cloud-contaminated water maps. The benefits of synthetic aperture radar images (e.g., Sentinel-1) for enhancing flood mapping under cloud cover conditions were also discussed. The method proposed in this paper provided an effective way for flood monitoring in cloudy and rainy scenarios, supporting emergency response and disaster management. The code and datasets used in this study have been made available online (https://github.com/dr-lizhiwei/SeamlessFloodMapper).

Coastal tidal wetlands are crucial for environmental and economic health, but facing threats from various environmental changes. Detecting changes of tidal wetlands is essential for promoting sustainable development in coastal areas. Despite extensive researches on tidal wetland changes, persistent challenges still exist. Firstly, the high similarity among tidal wetland types hinders the effectiveness of existing common indices. Secondly, many current methods, relying on hand-crafted features, are time-consuming and subject to personal biases. Thirdly, few studies effectively integrate multi-temporal and semantic information, leading to misinterpretations from environmental noise and tidal variations. In view of the abovementioned issues, we proposed a novel temporal-spectral-semantic-aware convolutional transformer network (TSSA-CTNet) for multi-class tidal wetland change detection. Firstly, to address spectral similarity among different tidal wetlands, we proposed a sparse second order feature construction (SSFC) module to construct more separable spectral representations. Secondly, to get more separable features automatically, we constructed temporal-spatial feature extractor (TSFE) and siamese semantic sharing (SiamSS) blocks to extract temporal-spatial-semantic features. Thirdly, to fully utilize semantic information, we proposed a center comparative label smoothing (CCLS) module to generate semantic-aware labels. Experiments in the Greater Bay Area, using Landsat data from 2000 to 2019, demonstrated that TSSA-CTNet achieved 89.20% overall accuracy, outperforming other methods by 3.75%–16.39%. The study revealed significant area losses in tidal flats, mangroves, and tidal marshes, decreased by 3148 hectares, 35 hectares, and 240 hectares, respectively. Among the cities in GBA, Zhuhai shows the most significant area loss with a total of 1626 hectares. TSSA-CTNet proves effective for multi-class tidal wetland change detection, offering valuable insights for tidal wetland protection.

Optical remote sensing of snow is challenged by the complex radiative transfer mechanism in alpine environments. The representation of topographic effects in interpreting satellite imagery of snow is still limited to inadequate analytical modelization. Here we develop a framework that explicitly solves multiple terrain reflections and generates the top-of-atmosphere (TOA) radiance of alpine snow by the modified four-stream radiative transfer theory. This framework comprises an atmosphere module, a terrain module and a surface spectra module relying on the approximate asymptotic radiative transfer (ART) model. In the terrain module, the iterative solution to multiple terrain reflections is facilitated with a viewshed calculating algorithm which identifies adjacent slopes and related geometric angles to derive terrain-reflected irradiance. The modeled TOA radiance is compared with Landsat-8/9 OLI, Sentinel-2A/B MSI and Terra MODIS radiance imagery. Experiments of several snow-covered mountainous regions in the Pamir area reveal that the TOA radiance modeling results agree well with satellite observations with reported $R^2$ $\ge 0.86$, though subject to the uncertainties due to complex topography and seasonality. The modeled terrain-reflected irradiance is verified with the ray-tracing software called LargE-Scale Remote Sensing Data and Image Simulation Framework (LESS), and reliable modeling performance is confirmed as $R^2$ values are $\ge 0.90$. This model framework allows for better interpreting the apparent spectra of alpine snow through the physically-based linkage with snow’s intrinsic properties and environmental conditions.

Crop type maps are essential in informing agricultural policy decisions by providing crucial data on the specific crops cultivated in given regions. The generation of crop type maps usually involves the collection of ground truth data of various crop species, which can be challenging at large scales. As an alternative to conventional field observations, street view images offer a valuable and extensive resource for gathering large-scale crop type ground truth through imaging the crops cultivated in the roadside agricultural fields. Yet our ability to systematically retrieve crop type labels at large scales from street view images in an operational fashion is still limited. The crop type retrieval is usually at the pixel level with uncertainty seldom considered. In our study, we develop a novel deep learning-based CropSight modeling framework to retrieve the object-based crop type ground truth by synthesizing Google Street View (GSV) and PlanetScope satellite images. CropSight comprises three key components: (1) A large-scale operational cropland field-view imagery collection method is devised to systematically acquire representative geotagged cropland field-view images of various crop types across regions in an operational manner; (2) UncertainFusionNet, a novel Bayesian convolutional neural network, is developed to retrieve high-quality crop type labels from collected field-view images with uncertainty quantified; (3) Segmentation Anything Model (SAM) is fine-tuned and employed to delineate the cropland boundary tailored to each collected field-view image with its coordinate as the point prompt using the PlanetScope satellite imagery. With four agricultural dominated regions in the US as study areas, CropSight consistently shows high accuracy in retrieving crop type labels of multiple dominated crop species (overall accuracy around 97 %) and in delineating corresponding cropland boundaries (F1 score around 92 %). UncertainFusionNet outperforms the benchmark models (i.e., ResNet-50 and Vision Transformer) for crop type image classification, showing an improvement in overall accuracy of 2–8 %. The fine-tuned SAM surpasses the performance of Mask-RCNN and the base SAM in cropland boundary delineation, achieving a 4–12 % increase in F1 score. The further comparison with the benchmark crop type product (i.e., cropland data layer (CDL)) indicates that CropSight is a promising alternative to crop type mapping products for providing high-quality, object-based crop type ground truth of diverse crop species at large scales. CropSight holds considerable promise to extrapolate over space and time for operationalizing large-scale object-based crop type ground truth retrieval in a near-real-time manner.

The CityGML Level of Detail 3 (LoD3), a widely adopted standard for three-dimensional (3D) city modeling, has been accessible for an extended period. However, its comprehensive implementation remains limited due to challenges such as insufficient automation and inconsistent data quality. This research introduces an innovative and fully automated framework aimed at urban-scale semantic building model reconstruction. The proposed framework addresses three critical challenges: (1) proposing facade layout graph model to formalize the geometry and topological relationships of semantic entities on building facades, thereby promoting the deduction of structural completeness and the reconstruction of semantic facade models; (2) establishing a mapping relationship between texture images, semantic entities, and building shells guided by the facade layout graph to ensure consistent correlations among the geometry, semantics, and topology of building models; (3) developing an efficient representation methodology for semantic building models utilizing a parameter set derived from the facade layout graph. The proposed framework has been successfully validated by reconstructing 8,681 buildings from three different locations in Berlin. The results demonstrate an outstanding reconstruction accuracy of 91%, with a time efficiency of only 3.42 s per building. Visual analysis further confirms that the framework effectively fulfills the application prerequisites of 3D GIS. The code of the proposed framework is available in the repository: https://github.com/wangyuefeng2017/LoD3Framework-.

This paper presents a catadioptric omnidirectional thermal odometry (COTO) system that estimates the six degrees of freedom (DoF) pose of a camera using only omnidirectional thermal images in visually degraded, fast-motion, and dynamic environments. First, we design and fabricate a central hyperbolic catadioptric omnidirectional thermal camera that captures surrounding thermal images with $360°$ horizontal field of view (FoV), and improve the omnidirectional camera model and calibration method to obtain high-precision camera intrinsic parameter. Second, we propose the epipolar curve constraint combining with omnidirectional thermal object detection to significantly reduce the interference of moving objects on pose estimation. Third, the implemented COTO pipeline consists of photometric calibration, dynamic region removal, tracking and mapping to overcome the drawbacks of photometric inconsistency and large distortion in omnidirectional thermal images. Experiments have been conducted on a total of 17 sequences of Lab, Outdoor and Driving, amounting to more than 60,000 omnidirectional thermal images of real environments. The experimental results indicate that the proposed COTO system has excellent localization accuracy and unparalleled robustness over the current state-of-the-art methods. The average localization accuracy measured by the absolute trajectory error (ATE) is less than 15 cm from the ground truth in both Lab and Outdoor sequences. In addition, COTO was the only system with complete and successful tracking in all sequences. The system can be used as an innovative localization solution, particularly in challenging environments with changes in ambient light, rapid vehicle motion, and moving object interference, which can be a difficult problem for visual odometry to solve.