ISPRS Journal of Photogrammetry and Remote Sensing最新文献

Accurate and timely information on the spatial distribution and areas of crop types is critical for yield estimation, agricultural management, and sustainable development. However, traditional crop classification methods often struggle to identify various crop types effectively due to their intricate spatiotemporal patterns and high training data demands. To address this challenge, we developed a Structure-aware Label eXpansion segmentation Network (StructLabX-Net) for diverse crop type mapping using limited point-annotated samples. StructLabX-Net features a backbone U-TempoNet, which combines CNNs and LSTM to explore intricate spatiotemporal patterns. It also incorporates multi-task weak supervision heads for edge detection and pseudo-label expansion, adding crucial structure and contextual insights. We tested the StructLabX-Net across three distinct regions in China, assessing over 10 crop types and comparing its performance against five popular classifiers based on multi-temporal Sentinel-2 images. The results showed that StructLabX-Net significantly outperformed RF, SVM, DeepCropMapping, Transformer, and patch-based CNN in identifying various crop types across three regions with sparse training samples. It achieved the highest overall accuracy and mean F1-score: 91.0% and 89.1% in Jianghan Plain, 91.5% and 90.7% in Songnen Plain, as well as 91.0% and 90.8% in Sanjiang Plain. StructLabX-Net demonstrated a particular advantage for those “hard types” characterized by limited samples and complex phenological features. Furthermore, ablation experiments highlight the crucial role of the “edge” head in guiding the model to accurately differentiate between various crop types with clearer class boundaries, and the “expansion” head in refining the understanding of target crops by providing extra details in pseudo-labels. Meanwhile, combining our backbone U-TempoNet with multi-task weak supervision heads exhibited superior results of crop type mapping than those derived by other segmentation models. Overall, StructLabX-Net maximizes the utilization of limited sparse samples from field surveys, offering a simple, cost-effective, and robust solution for accurately mapping various crop types at large scales. The code will be publicly available at https://github.com/BruceKai/StructLabX-Net.

Classification of urban land use and land cover is vital to many applications, and naturally becomes a popular topic in remote sensing. The finite information carried by unimodal data, the compound land use types, and the poor signal-noise ratio caused by restricted weather conditions would inevitably lead to relatively poor classification performance. Recently in remote sensing society, multimodal data fusion with deep learning technology has gained a great deal of attention. Existing research exhibit integration of multimodal data at a single level, while simultaneously lacking exploration of the immense potential provided by popular transformer and CNN structures for effectively leveraging multimodal data, which may fall into the trap that makes the information fusion inadequate. We introduce SoftFormer, a novel network that synergistically merges the strengths of CNNs with transformers, as well as achieving multi-level fusion. To extract local features from images, we propose an innovative mechanism called Interior Self-Attention, which is seamlessly integrated into the backbone network. To fully exploit the global semantic information from both modalities, in the feature-level fusion, we introduce a joint key–value learning fusion approach to integrate multimodal data within a unified semantic space. The decision and feature level information are simultaneously integrated, resulting in a multi-level fusion transformer network. Results on four remote sensing datasets show that SoftFormer is able to achieve at least 1.32%, 0.7%, and 0.99% performance improvement in overall accuracy, kappa index, and mIoU, compared to other state-of-the-art methods, the ablation studies show that multimodal fusion outperforms the unimodal data on urban land cover and land use classification, the highest overall accuracy, kappa index as well as mIoU improvement can be up to 5.71%, 10.32% and 7.91%, and the proposed modules are able to boost performance to some extent, even with cloud cover. Code will be publicly available at https://github.com/rl1024/SoftFormer.

Understanding the spatial and temporal trends of burned areas (BA) on a global scale offers a comprehensive view of the underlying mechanisms driving fire incidence and its influence on ecosystems and vegetation recovery patterns over extended periods. Such insights are invaluable for modeling fire emissions and the formulation of strategies for post-fire rehabilitation planning.

Previous research has provided strong evidence that current global BA products derived from coarse spatial resolution data underestimates global burned areas. Consequently, there is a pressing need for global high-resolution BA products. Here, we present an automatic global burned area mapping algorithm (Sentinel2BAM) based on Sentinel-2 Level-2A imagery combined with Visible Infrared Imaging Radiometer Suite (VIIRS) and Moderate Resolution Imaging Spectrometer (MODIS) active fire data. The algorithm employs a Random Forest Model trained by active fires to predict BA probabilities in each 5-day Normalized Burn Ratio (NBR) index-based temporal composites. In a second step, a time-series and object-based analysis of the estimated BA probabilities allows burned areas to be detected on a quarterly basis. The algorithm was implemented in Google Earth Engine (GEE) and applied to 576 Sentinel-2 tiles corresponding to 2019, distributed globally, to assess its ability to map burned areas across different ecosystems. Two validation sources were employed: 21 EMSR Copernicus Emergency Service perimeters obtained using high spatial resolution (<10 m) data (EMSR21) located in the Mediterranean basin and 50 20x20 km global samples selected by stratified sampling with Sentinel-2 at 10 m spatial resolution (GlobalS50). Additionally, 105 Landsat-based long sample units (GlobalL105), were employed to compare the performance of the Sentinel2BAM algorithm against the FIRECCI51 and MCD64A1 global products. Overall accuracy metrics for the Sentinel2BAM algorithm, derived from validation sources highlight higher commission (CE) than omission (OE) errors (CE=10.3 % and OE=7.6 % when using EMSR21 as reference, CE=18.9 % and OE=9.5 % when using Global S50 as reference), while GlobalL105-based inferenced global comparison metrics show similar patterns (CE=22.5 % and OE=13.4 %). Results indicate differences across ecosystems: forest fires in tropical and temperate biomes exhibit higher CE, mainly due to confusion between burned areas and croplands. According to GlobalL105, Sentinel2BAM shows greater accuracy globally (CE=22.5 %, OE=13.4 %) compared to FIRECCI51 (CE=20.8 %, OE=46.5 %) and MCD64A1 (CE=17.5 %, OE=53.1 %), substantially improving the detection of small fires and thereby reducing omission errors. The strengths and weaknesses of the algorithm are thoroughly addressed, demonstrating its potential for global application.

Thin cloud disturbs the observation of optical sensors, thus reducing the quality of optical remote sensing images and limiting the subsequent applications. However, the reliance of the existing thin cloud correction methods on the assistance of in-situ parameters, prior assumptions, massive paired data, or special bands severely limits their generalization. Moreover, due to the inadequate consideration of cloud characteristics, these methods struggle to obtain accurate results with complex degradations. To address the above two problems, a thin cloud blind correction (TC-BC) method coupling a cloudy image imaging model and a feature separation network (FSNet) module is proposed in this paper, based on an unsupervised self-training framework. Specifically, the FSNet module takes the independence and obscure boundary characteristics of the cloud into account to improve the correction accuracy with complex degradations. The FSNet module consists of an information interaction structure for exchanging the complementary features between cloud and ground, and a spatially adaptive structure for promoting the learning of the thin cloud distribution. Thin cloud correction experiments were conducted on an unpaired blind correction dataset (UBCSet) and the proposed TC-BC method was compared with three traditional methods. The visual results suggest that the proposed method shows obvious advantages in information recovery for thin cloud cover regions, and shows a superior global consistency between cloudy regions and clear regions. The TC-BC method also achieves the highest peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM). The FSNet module in the TC-BC method is also proven to be effective. The FSNet module can achieve a superior precision when compared with five other deep learning networks in cloud-ground separation performance. Finally, extra experimental results show that the TC-BC method can be applied to different cloud correction scenarios with varied cloud coverage, surface types, and image scales, demonstrating its generalizability. Code: https://github.com/Liying-Xu/TCBC.

Performing multiple scans is necessary to cover an entire scene of interest, making multi-station registration adjustment a critical task in terrestrial laser scanner data processing. Existing methods either rely on pair-wise adjustment, which leads to drift accumulation and lacks global consistency, or provide an approximate solution based on a linearized model, sacrificing statistical optimality. In this study, using a multi-station stacking model, we propose a method that provides two different nonlinear least-squares (LS) solutions to this problem. We first demonstrate how a nonlinear Baarda’s S-transformation can be used to transform solutions that share the same optimal network configuration. Then, two practically meaningful LS solutions are introduced, i.e., the trivial minimal-constraints solution and the partial nearest solution. Most importantly, we derive a truncated Gauss–Newton iterative scheme to obtain numerically exact solutions to the corresponding nonlinear rank-deficient problem. We validate our method with three real-world examples, demonstrating that (1) global consistency is maintained with no drift accumulation, and (2) our nonlinear solution outperforms the approximate linearized solution. Code and data are available at https://github.com/huyuchn/Multi-station-registration.

Multi-temporal optical remote sensing images acquired from cross-sensor platforms often show significant radiometric differences, posing challenges when mosaicking images. These challenges include inconsistent global radiometric tones, unsmooth local radiometric transitions, and visible seamlines. In this paper, to address these challenges, we propose a two-stage approach for global and local radiometric normalization (RN) using joint block adjustment and variational optimization. In the first stage, a block adjustment based global RN (BAGRN) model is established to simultaneously perform global RN on all the images, eliminating global radiometric differences and achieving overall radiometric tonal consistency. In the second stage, a variational optimization based local RN (VOLRN) model is introduced to address the remaining local radiometric differences after global RN. The VOLRN model applies local RN to all the image blocks within a unified energy function and imposes the $l_1$ norm constraint on the data fidelity term, providing the model with a more flexible local RN capability to radiometrically normalize the intersection and transition areas of the images. Therefore, the local radiometric discontinuities and edge artifacts can be eliminated, resulting in natural and smooth local radiometric transitions. The experimental results obtained on five challenging datasets of cross-sensor and multi-temporal remote sensing images demonstrate that the proposed approach excels in both visual quality and quantitative metrics. The proposed approach effectively eliminates global and local radiometric differences, preserves image gradients well, and has high processing efficiency. As a result, it outperforms the state-of-the-art RN approaches.

3D point cloud semantic segmentation, a pivotal task in fields such as autonomous driving and urban planning, confronts the challenge of performance degradation under adverse weather conditions. Current methodologies primarily focus on optimal weather scenarios, leaving a significant gap in handling various environmental adversities like fog, rain, and snow. To bridge this gap, we propose a comprehensive deep learning framework featuring unique components — an Adaptive Feature Normalization Module (AFNM) for effective normalization and calibration of features, a Dual-Attention Fusion Module (DAFM) for integrating cross-domain features, and a Proxy Label Generation Module (PLGM) for generating reliable proxy labels within the domain. Utilizing the SemanticKITTI and SynLiDAR datasets as source domains and the SemanticSTF dataset as the target domain, our model has been rigorously evaluated under varying weather conditions. When trained on the SemanticKITTI dataset as the source domain with the SemanticSTF dataset as the target, our approach surpasses the current state-of-the-art models by a margin of 6.2% in terms of overall mean Intersection over Union (mIoU) scores. Similarly, with the SynLiDAR dataset as the source and SemanticSTF as the target, our performance exceeds the best existing models by 3.4% in mIoU. These results substantiate the efficacy of our model in advancing the field of 3D semantic segmentation under diverse weather conditions, showcasing its notable robustness and superiority. The code is available at https://github.com/J2DU/WADG-PointSeg.

The time series data of Normalized Difference Vegetation Index (NDVI) is crucial for monitoring changes in terrestrial vegetation. Existing reconstruction methods encounter challenges in areas prone to clouds, primarily due to inadequate utilization of spatial, temporal, periodic, and multi-sensor information, as well as a lack of physical interpretations. This frequently results in limited model performance or the omission of spatial details when predicting scenarios involving land cover changes. In this study, we propose a novel approach named Residual (Re) Constraints (Co) fusion-and-fit (ReCoff), consisting of two steps: ReCoF fusion (F) and Savitzky-Golay (SG) fit. This approach addresses the challenges of reconstructing 30 m Landsat NDVI time series data in cloudy regions. The fusion-fit process captures land cover changes and maps them from MODIS to Landsat using a deep learning model with residual constraints, while simultaneously integrating multi-dimensional, multi-sensor, and long time-series information. ReCoff offers three distinct advantages. First, the fusion results are more robust to land cover change scenarios and contain richer spatial details (RMSE of 0.091 vs. 0.101, 0.164, and 0.188 for ReCoF vs. STFGAN, FSDAF, and ESTARFM). Second, ReCoff improves the effectiveness of reconstructing dense time-series data (2016–2020, 16-day interval) in cloudy areas, whereas other methods are more susceptible to the impact of prolonged data gaps. ReCoff achieves a correlation coefficient of 0.84 with the MODIS reference series, outperforming SG (0.28), HANTS (0.32), and GF-SG (0.48). Third, with the help of the GEE platform, ReCoff can be applied over large areas (771 km × 634 km) and long-time scales (bimonthly intervals from 2000 to 2020) in cloudy regions. ReCoff demonstrates potential for accurately reconstructing time-series data in cloudy areas.

The multi-spectral imagery based Satellite Derived Bathymetry (SDB) provides an efficient and cost-effective approach for acquiring bathymetry data of nearshore shallow water regions. Compared with conventional pixelwise inversion models, Deep Learning (DL) models have the theoretical capability to encompass a broader receptive field, automatically extracting comprehensive spatial features. However, enhancing spatial features by increasing the input size escalates computational complexity and model scale, challenging the hardware. To address this issue, we propose the Multiple Scale Resolution Fusion Model (MuSRFM), a novel DL-based SDB model, to integrate information of varying scales by utilizing temporally fused Sentinel-2 L2A multi-spectral imagery. The MuSRFM uses a Multi-scale Center-aligned Hierarchical Resampler (MCHR) to composite large-scale multi-spectral imagery into hierarchical scale resolution representations since the receptive field gradually narrows its focus as the spatial resolution decreases. Through this strategy, the MuSRFM gains access to rich spatial information while maintaining efficiency by progressively aggregating features of different scales through the Cropped Aligned Fusion Module (CAFM). We select St. Croix (Virgin Islands) as the training/testing dataset source, and the Root Mean Square Error (RMSE) obtained by the MuSRFM on the testing dataset is 0.8131 m (with a bathymetric range of 0–25 m), surpassing the machine learning based models and traditional semi-empirical models used as the baselines by over 35 % and 60 %, respectively. Additionally, multiple island areas worldwide, including Vieques, Oahu, Kauai, Saipan and Tinian, which exhibit distinct characteristics, are utilized to construct a real-world dataset for assessing the generalizability and transferability of the proposed MuSRFM. While the MuSRFM experiences a degradation in accuracy when applied to the diverse real-world dataset, it outperforms other baseline models considerably. Across various study areas in the real-world dataset, its RMSE lead over the second-ranked model ranges from 6.8 % to 38.1 %, indicating its accuracy and generalizability; in the Kauai area, where the performance is not ideal, a significant improvement in accuracy is achieved through fine-tuning on limited in-situ data. The code of MuSRFM is available at https://github.com/qxm1995716/musrfm.

Snow depth (SD) is a crucial property of snow, its spatial and temporal variation is important for global change, snowmelt runoff simulation, disaster prediction, and freshwater storage estimation. Polarimetric Synthetic Aperture Radar (PolSAR) can precisely describe the backscattering of the target and emerge as an effective tool for SD retrieval. The backscattering component of dry snow is mainly composed of volume scattering from the snowpack and surface scattering from the snow-ground interface. However, the existing method for retrieving SD using PolSAR data has the problems of over-reliance on in-situ data and ignoring surface scattering from the snow-ground interface. We proposed a novel SD retrieval method for PolSAR data by fully considering the primary backscattering components of snow and through multi-parameter estimation to solve the snow backscattering model. Firstly, a snow backscattering model was formed by combining the small permittivity volume scattering model and the Michigan semi-empirical surface scattering model to simulate the different scattering components of snow, and the corresponding backscattering coefficients were extracted using the Yamaguchi decomposition. Then, the snow permittivity was calculated through generalized volume parameters and the extinction coefficient was further estimated through modeling. Finally, the snow backscattering model was solved by these parameters to retrieve SD. The proposed method was validated by Ku-band UAV SAR data acquired in Altay, Xinjiang, and the accuracy was evaluated by in-situ data. The correlation coefficient, root mean square error, and mean absolute error are 0.80, 4.49 cm, and 3.95 cm, respectively. Meanwhile, the uncertainties generated by different SD, model parameters estimation, solution method, and underlying surface are analyzed to enhance the generality of the proposed method.