A cost-effective and robust mapping method for diverse crop types using weakly supervised semantic segmentation with sparse point samples

Abstract

Accurate and timely information on the spatial distribution and areas of crop types is critical for yield estimation, agricultural management, and sustainable development. However, traditional crop classification methods often struggle to identify various crop types effectively due to their intricate spatiotemporal patterns and high training data demands. To address this challenge, we developed a Structure-aware Label eXpansion segmentation Network (StructLabX-Net) for diverse crop type mapping using limited point-annotated samples. StructLabX-Net features a backbone U-TempoNet, which combines CNNs and LSTM to explore intricate spatiotemporal patterns. It also incorporates multi-task weak supervision heads for edge detection and pseudo-label expansion, adding crucial structure and contextual insights. We tested the StructLabX-Net across three distinct regions in China, assessing over 10 crop types and comparing its performance against five popular classifiers based on multi-temporal Sentinel-2 images. The results showed that StructLabX-Net significantly outperformed RF, SVM, DeepCropMapping, Transformer, and patch-based CNN in identifying various crop types across three regions with sparse training samples. It achieved the highest overall accuracy and mean F1-score: 91.0% and 89.1% in Jianghan Plain, 91.5% and 90.7% in Songnen Plain, as well as 91.0% and 90.8% in Sanjiang Plain. StructLabX-Net demonstrated a particular advantage for those “hard types” characterized by limited samples and complex phenological features. Furthermore, ablation experiments highlight the crucial role of the “edge” head in guiding the model to accurately differentiate between various crop types with clearer class boundaries, and the “expansion” head in refining the understanding of target crops by providing extra details in pseudo-labels. Meanwhile, combining our backbone U-TempoNet with multi-task weak supervision heads exhibited superior results of crop type mapping than those derived by other segmentation models. Overall, StructLabX-Net maximizes the utilization of limited sparse samples from field surveys, offering a simple, cost-effective, and robust solution for accurately mapping various crop types at large scales. The code will be publicly available at https://github.com/BruceKai/StructLabX-Net.