Reference-based image super-resolution of hyperspectral and red-green-blue image for determination of wheat kernel quality using deep learning networks

Abstract

In the process of cultivation and harvest, wheat kernel quality is highly susceptible to various factors, such as disease, mildew, atrophy and impurities, and detection of kernel quality is essential to avoid hazard proliferation, facilitate product grading, and ensure food safety. Possessing abundant image and spectral characteristics, hyperspectral imaging (HSI) has gained impressive achievements in kernel quality analysis, but its low spatial resolution limits its detection accuracy. In this study, reference-based image super-resolution (RefSR) of HSI and Red-Green-Blue image was adopted to improve resolution to determine wheat kernel quality using deep learning networks. Firstly, RefSR was conducted by the improved transformer network with dual-branch feature extraction and weighted fusion operation and achieved excellent RefSR with significant resolution improvement, peak signal to noise ratio of 35.521 and structural similarity index of 0.97, outweighing the existing state-of-the-art networks. Then, the reflectance images (RIs) of effective wavelengths (EWs) from generated HSI images were combined with the residual network with a spatial, channel attention and multi-scale residual to determine wheat kernel quality. Precise analysis was achieved with the accuracy in calibration, validation and prediction sets of 100.00%, 95.26% and 92.78%. RefSR provides a novel and efficient approach for obtaining HSI images of high spatial resolution and facilitates the application of HSI in analysis of crop kernels. RIs of several sporadic EWs can be easily acquired and processed, achieving field and rapid kernel detection. Therefore, the proposed method furnishes the efficient, accurate and applicable determination of wheat kernel quality.