A temporal attention-based multi-scale generative adversarial network to fill gaps in time series of MODIS data for land surface phenology extraction

Abstract

High-quality and continuous satellite data are essential for land surface studies such as monitoring of land surface phenology, but factors such as cloud contamination and sensor malfunction degrade the quality of remote sensing images and limit their utilization. Filling gaps and recovering missing information in time series of remote sensing images are vital for a wide range of downstream applications, such as land surface phenology extraction. Most existing gap-filling and cloud removal methods focus on individual or multi-temporal image reconstruction, but struggle with continuous and overlapping missing areas in time series data. In this study, we propose a Temporal Attention-Based Multi-Scale Generative Adversarial Network (TAMGAN) to reconstruct time series of Moderate Resolution Imaging Spectroradiometer (MODIS) data. TAMGAN leverages a Generative Adversarial Network (GAN) with a 3-dimensional Convolution Neural Networks (3DCNN) in its generator to reconstruct the missing areas in the annual time series of remote sensing images simultaneously. The temporal attention blocks are designed to capture the changing trends of surface reflectance over time. And multi-scale feature extraction and progressive concatenation are introduced to enhance spectral consistency and provide detailed texture information. Experiments are carried out on MOD09A1 products to evaluate the performance of the proposed network. The results show that TAMGAN outperformed the comparison methods across various evaluation metrics, particularly in handling large and continuous missing areas in the time series. Furthermore, we showcase an example of downstream application by extracting phenological information from the gap-filled products. By effectively filling gaps and removing clouds, our method offers spatial-temporal continuous MODIS surface reflectance data, benefiting downstream applications such as phenology extraction and highlighting the potential of artificial intelligence technique in remote sense data processing.