ISPRS Journal of Photogrammetry and Remote Sensing最新文献

Accurate mapping of winter wheat provides essential information for food security and ecosystem protection. Deep learning approaches have achieved promising crop discrimination performance based on multitemporal satellite imagery. However, due to the high dimensionality of the data, sequential relations, and complex semantic information in time-series imagery, effective methods that can automatically capture temporal-spatial features with high separability and generalizability have received less attention. In this study, we proposed a U-shaped CNN-Transformer hybrid framework based on an attention mechanism, named the U-Temporal-Spatial-Transformer network (UTS-Former), for winter wheat mapping using Sentinel-2 imagery. This model includes an “encoder-decoder” structure for multiscale information mining of time series images and a temporal-spatial transformer module (TST) for learning comprehensive temporal sequence features and spatial semantic information. The results obtained from two study areas indicated that our UTS-Former achieved the best accuracy, with a mean MCC of 0.928 and an F1-score of 0.950, and the results of different band combinations also showed better performance than other popular time-series methods. We found that the MCC (MCC/All) of the UTS-Former using only RGB bands decreased by 4.53 %, while it decreased by 13.36 % and 35.02 % for UNet2d-LSTM and CNN-BiLSTM, respectively, compared with that of all the band combinations. The comparison demonstrated that the proposed UTS-Former could capture more global temporal-spatial information from winter wheat fields and achieve greater precision in terms of local details than other methods, resulting in high-quality mapping. The analysis of attention scores for the available acquisition dates revealed significant contributions of both beginning and ending growth images in winter wheat mapping, which is valuable for making appropriate selections of image dates. These findings suggest that the proposed approach has great potential for accurate, cost-effective, and high-quality winter wheat mapping.

Three-dimensional (3D) mapping is important to achieve early warning for construction safety and support the long-term safety maintenance of tunnels. However, generating 3D point cloud maps of excavation tunnels that tend to be deficient in features, have rough lining structures, and suffer from dynamic construction interference, can be a challenging task. In this paper, we propose a novel legged robot-aided 3D tunnel mapping method to address the influence of point clouds in the mapping phase. First, a method of kinematic model construction that integrates information from both the robot’s motors and the inertial measurement unit (IMU) is proposed to correct the motion distortion of point clouds. Then, a residual compensation model for unreliable regions (abbreviated as the URC model) is proposed to eliminate the inherent alignment errors in the 3D structures. The structural regions of a tunnel are divided into different reliabilities using the K-means method, and an inherent alignment metric is compensated based on region residual estimation. The compensated alignment metric is then incorporated into a rotation-guided anomaly consistency detection (RAD) model. An isolation forest-based anomaly consistency indicator is designed to remove anomalous light detection and ranging (LiDAR) points and reduce sensor noise caused by ultralong distances. To verify the proposed method, we conduct numerous experiments in three tunnels, namely, a drilling and blasting tunnel, a TBM tunnel, and an underground pedestrian tunnel. According to the experimental results, the proposed method achieves 0.84 ‰, 0.40 ‰, and 0.31 ‰ closure errors (CEs) for the three tunnels, respectively, and the absolute map error (AME) and relative map error (RME) are approximately 1.45 cm and 0.57 %, respectively. The trajectory estimation and mapping errors of our method are smaller than those of existing methods, such as FAST-LIO2, Faster-LIO and LiLi-OM. In addition, ablation tests are conducted to further reveal the roles of the different models used in our method for legged robot-aided 3D mapping in tunnels.

Visual question answering (VQA) is an important and challenging multimodal task in computer vision and photogrammetry. Recently, efforts have been made to bring the VQA task to aerial images, due to its potential real-world applications in disaster monitoring, urban planning, and digital earth product generation. However, the development of VQA in this domain is restricted by the huge variation in the appearance, scale, and orientation of the concepts in aerial images, along with the scarcity of well-annotated datasets. In this paper, we introduce a new dataset, HRVQA, which provides a collection of 53,512 aerial images of 1024 × 1024 pixels and semi-automatically generated 1,070,240 QA pairs. To benchmark the understanding capability of VQA models for aerial images, we evaluate the recent methods on the HRVQA dataset. Moreover, we propose a novel model, GFTransformer, with gated attention modules and a mutual fusion module. The experiments show that the proposed dataset is quite challenging, especially the specific attribute-related questions. Our method achieves superior performance in comparison to the previous state-of-the-art approaches. The dataset and the source code are released at https://hrvqa.nl/.

Cloud coverage poses a significant challenge to optical image interpretation, degrading ground information on Earth’s surface. Synthetic aperture radar (SAR), with its ability to penetrate clouds, provides supplementary information to optical data. However, existing optical-SAR fusion methods predominantly focus on cloud-free scenarios, neglecting the practical challenge of semantic segmentation under cloudy conditions. To tackle this issue, we propose CloudSeg, a novel framework tailored for land cover mapping in the presence of clouds. It addresses the challenges posed by cloud cover from two aspects: reducing semantic ambiguity in areas of the cloudy image that are obscured by clouds and enhancing effective information in the unobstructed portions. Specifically, CloudSeg employs a multi-task learning strategy to simultaneously handle low-level visual task and high-level semantic understanding task, mitigating the semantic ambiguity caused by cloud cover by acquiring discriminative features through an auxiliary cloud removal task. Additionally, CloudSeg incorporates a knowledge distillation strategy, which utilizes the knowledge learned by the teacher network under cloud-free conditions to guide the student network to overcome the interference of cloud-covered areas, enhancing the valuable information from the unobstructed parts of cloud-covered images. Extensive experiments conducted on two datasets, M3M-CR and WHU-OPT-SAR, demonstrate the effectiveness and superiority of the proposed CloudSeg method for land cover mapping under cloudy conditions. Specifically, CloudSeg outperforms the state-of-the-art competitors by 3.16% in terms of mIoU on M3M-CR and by 5.56% on WHU-OPT-SAR, highlighting its substantial advantages for analyzing regions frequently obscured by clouds. Codes are available at https://github.com/xufangchn/CloudSeg.

Over the past two decades, a major advance that enabled airborne bathymetric lidar to benefit a much wider range of marine science applications was the development of procedures for creating seafloor reflectance mosaics from recorded intensity data. It was recognized that intensity data, derived from the amplitudes of laser returns from the seafloor, contained information related to seafloor albedo and composition. However, the raw intensity data were also found to be related to a number of nuisance parameters, such that, when grided, they exhibited discontinuities, seamlines and other artifacts, hindering their use in benthic habitat mapping. These realizations led to the development of tools and workflows for correcting lidar intensity data to produce seamless seafloor reflectance mosaics. At present, an opportunity exists for another major advance in airborne bathymetric lidar by utilizing not only intensity data, but a large suite of waveform features that describe the shape of the return signal from the seafloor, to characterize benthic habitats and perform ecological assessments. However, similar to raw intensity data, other waveform features exhibit salient discontinuities, seamlines, and other artifacts, if uncorrected. Furthermore, in contrast to the case of intensity data, little work has been done on correction of an entire suite of waveform features to create a set of seamless seafloor mosaics. This study aims to address this need through a novel normalization method that integrates two image blending techniques: Gaussian weighted color matching and Laplacian pyramid blending. The proposed approach, Simultaneous Invariant Normalization of Waveform Features (SINWav), is designed to be invariant to the type of input waveform features, such that feature-specific tuning is unnecessary. To handle vast amounts of data efficiently, we developed a memory-efficient sparse matrix representation. The methods were applied to bathymetric lidar data from Saipan containing 16 different waveform features. Both visual assessments and quantitative analyses using quality metrics indicated that the proposed approach outperforms results derived from raw data and conventional linear transform.

Accurate cloud and shadow detection is a crucial prerequisite for optical remote sensing image analysis and application. Multi-temporal-based cloud and shadow detection methods are a preferable choice to detect clouds in complex scenes (e.g., thin clouds, broken clouds and clouds with interference from artificial surfaces with high reflectivity). However, such methods commonly require cloud-free reference images, and this may be difficult to achieve in time-series data since clouds are often prevalent and of varying spatial distribution in optical remote sensing images. Furthermore, current multi-temporal-based methods have limited feature extraction capability and rely heavily on prior assumptions. To address these issues, this paper proposes a Siamese network (Siamnet) for cloud and shadow detection based on Time-Series cloudy Images, namely TSI-Siamnet, which consists of two steps: 1) low-rank and sparse component decomposition of time-series cloudy images is conducted to construct a composite reference image to cope with dynamic changes in the cloud distribution in time-series images; 2) an extended Siamnet with optimal difference calculation module (DM) and multi-scale difference features fusion module (MDFM) is constructed to extract reliable disparity features and alleviate semantic information feature dilution during the decoder part. TSI-Siamnet was tested extensively on seven land cover types in the well-known Landsat 8 Biome dataset. Compared to six state-of-the-art methods (including four deep learning-based methods and two classical non-deep learning-based methods), TSI-Siamnet produced the best performance with an overall accuracy of 95.05% and MIoU of 84.37%. In three more challenging experiments, TSI-Siamnet showed enhanced detection of thin and broken clouds and greater anti-interference to highly reflective surfaces. TSI-Siamnet provides a novel strategy to explore comprehensively the valid information in time-series cloudy images and integrate the extracted spectral-spatial–temporal features for reliable cloud and shadow detection.

Changes in policy and new plans can significantly influence land use and trigger land use change in the long term. The data for pre- and post-policy implementation is necessary to assess the specific policy’s impact on land use. In the early nineties, Germany started promoting renewable energy production, including bioenergy, which changed the agricultural landscape. Remote sensing (RS) image-based machine learning models can be beneficial for mapping agricultural land use in the present and the past. However, machine learning classification models trained on RS data from specific training sites and time may not be able to predict data for unknown sites and unknown temporal points due to changes in crop phenology, field features, or ecological site circumstances because most of the models are limited in their performances according to variations of the training data set. Therefore, this study aims to assess the spatial–temporal transferability of Landsat-based agricultural land use type classification. The study was developed to map agricultural land cover (5 classes: maize, grasslands, summer crops, winter crops, and mixed crops) in two regions in Germany (North Hesse and Weser-Ems) between the years 2010 and 2018 using Landsat archive data (i.e., Landsat 5, 7, and 8). Two machine learning models (random forest − RF and 2D convolution neural network – 2DCNN) were trained and evaluated according to no transferability (reference) scenario and three spatial–temporal scenarios using mF1 and class level F1 values. Three model transferability scenarios were evaluated: a) temporal – S1, b) spatial – S2, and c) spatiotemporal – S3. The reference scenario, without transferability, achieved an overall accuracy of 89.1% and a macro F1 score of 0.74 for RF and 89.9% and 0.75 for CNN, respectively. Under three transferability scenarios (S1, S2, and S3), the macro F1 scores decreased to 0.67, 0.66, and 0.62 for RF, and 0.68, 0.62, and 0.58 for CNN, respectively. The dissimilarity between the data employed to train the model and data from the new domain indicated a clear link that could explain the reduction in model predictability. Moreover, the performance degradation could be attributed to the disparity in environmental, climatic, and crop calendar conditions between the two domains. Understanding the extent of model performance degradation during transferability is crucial for developing effective strategies to mitigate these issues and enhance the generalisability of machine learning models for agriculture land cover mapping.

The increasing availability of point cloud acquisition techniques makes it possible to significantly increase 3D observation capacity by the registration of multi-sensor, multi-platform, and multi-temporal point clouds. However, there are geometric heterogeneities (point density variations and point distribution differences), small overlaps (30 % ∼ 50 %), and large data amounts (a few millions) among these large-scale heterogeneous point clouds, which pose great challenges for effective and efficient registration. In this paper, considering the structural representation capacity of graph model, we propose an incremental registration method for large-scale heterogeneous point clouds by hierarchical graph matching. More specifically, we first construct a novel graph model to discriminatively and robustly represent heterogeneous point clouds. In addition to conventional nodes and edges, our graph model particularly designs discriminative and robust feature descriptors for local node description and captures spatial relationships from both locations and orientations for global edge description. We further devise a matching strategy to accurately estimate node matches for our graph models with partial even small overlaps. This effectiveness benefits from the comprehensiveness of node and edge dissimilarities and the constraint of geometric consistency in the optimization objective. On this basis, we design a coarse-to-fine registration framework for effective and efficient point cloud registration. In this incremental framework, graph matching is hierarchically utilized to achieve sparse-to-dense point matching by global extraction and local propagation, which provides dense correspondences for robust coarse registration and predicts overlap ratio for accurate fine registration, and also avoids huge computation costs for large-scale point clouds. Extensive experiments on one benchmark and three changing self-built datasets with large scales, outliers, changing densities, and small overlaps show the excellent transformation and correspondence accuracies of our registration method for large-scale heterogeneous point clouds. Compared to the state-of-the-art methods (i.e., TrimICP, CoBigICP, GROR, VPFBR, DPCR, and PRR), our registration method performs approximate even higher efficiency while achieves an improvement of 33 % − 88 % regarding registration accuracy (OE).

Traditional approach from source image to application vectors in building extraction needs additional complex regularization of converted intermediate raster results. While in conversion, the lost detailed artifacts, unnecessary nodes, and messy paths would be labor-intensive to repair errors and topological issues, even aside the inherent problems of blob-like objects and blurry, jagged edges in first-stage extraction. This research explores new graph convolution-driven solution, the spatial-cognitive shaping model (SCShaping), to directly access vectorization form of individual buildings through spatial cognitive approximation to coordinates that form building boundaries. To strengthen graph nodes expressivity, this method enriches topological feature embedding travelling along the model architecture along with features contributed from convolutional neural network (CNN) extractor. To stimulate the neighboring aggregation in graphs, Graph-Encoder-Decoder mechanism is introduced to augment feature reuse integrating complementary graph convolution layers. The strong embedding guarantees effective feature tapping and the robust structure guarantees the feature mining. Comparative studies have been conducted between the proposed approach with five other methods on three challenging datasets. The results demonstrate the proposed approach yields unanimous and significant improvements in mask-wise metrics, which evaluate object integrity and accuracy, as well as edge-wise metrics, which assess contour regularity and precision. The outperformance also indicates better multi-scale object adaptability of SCShaping. The obtain-and-play SCShaping commands a pleasurable implementation way to advance ideal man–machine collaboration.

Weather extremes affect crop production. Remote sensing can help to detect crop damage and estimate lost yield due to weather extremes over large spatial extents. We propose a novel scalable method to predict in-season yield losses at the sub-field level and attribute these to weather extremes. To assess our method’s potential, we conducted a proof-of-concept case study on winter cereal paddocks in South Australia using data from 2017 to 2022. To detect crop growth anomalies throughout the growing season, we aligned a two-band Enhanced Vegetation Index (EVI2) time series from Sentinel-2 with thermal time. The deviation between the expected and observed EVI2 time series was defined as the Crop Damage Index (CDI). We assessed the performance of the CDI within specific phenological windows to predict yield loss. Finally, by comparing instances of substantial increase in CDI with different extreme weather indicators, we explored which (combinations of) extreme weather events were likely responsible for the experienced yield reduction. We found that the use of thermal time diminished the temporal deviation of EVI2 time series between years, resulting in the effective construction of typical stress-free crop growth curves. Thermal-time-based EVI2 time series resulted in better prediction of yield reduction than those based on calendar dates. Yield reduction could be predicted before grain-filling (approximately two months before harvest) with an R² of 0.83 for wheat and 0.91 for barley. Finally, the combined analysis of CDI curves and extreme weather indices allowed for timely detection of weather-related causes of crop damage, which also captured the spatial variations of crop damage attribution at sub-field level. The proposed framework provides a basis for early warning of crop damage and attributing the damage to weather extremes in near real-time, which should help to adopt appropriate crop protection strategies.