Enhancing Winter Wheat Yield Estimation With a CNN-Transformer Hybrid Framework Utilizing Multiple Remotely Sensed Parameters

Abstract

To address insufficient feature extraction due to individual deep learning models’ limitations in capturing local and global features, and the tendency to underestimate high yields and overestimate low yields, a new deep learning model called convolutional neural network (CNN)-transformer with serial connection (CNN-transformers) was introduced to estimate winter wheat yield by combining the local feature extraction strengths of CNNs with the global information extraction abilities of transformer networks utilizing self-attention mechanisms. The remote sensing technology included temperature and spectral response indicators; the vegetation temperature condition index (VTCI), leaf area index (LAI), and fraction of photosynthetically active radiation (FPAR) aggregated over ten-day periods. Compared to the CNN model, the transformer model, the CNN-transformer model with parallel connection (CNN-transformerp), and the transformer-CNN model with serial connection (transformer-CNNs), the CNN-transformers achieved higher accuracy in estimating winter wheat yield ( ${R} ^{2} =0.70$ , RMSE =420.39 kg/ha, MAPE =7.65%), which was capable of extracting more information related to yield from various remotely sensed parameters and addressing the problems of high yield underestimation and low yield overestimation observed. The robustness and generalization of the CNN-transformers were further assessed through the fivefold cross-validation and the leave-one-year-out methods. In addition, utilizing the CNN-transformers, the study uncovered the cumulative impact of the winter wheat growth period, examined how incrementally adding data at ten-day intervals affects yield estimation, and assessed the model’s proficiency in depicting growth accumulation during the growth process. The findings indicated that the model well identified the crucial growth phase of winter wheat between late March and early May.