Real-time lettuce-weed localization and weed severity classification based on lightweight YOLO convolutional neural networks for intelligent intra-row weed control

Competition for nutrients between intra-row weeds and cultivated vegetables is a major contributor to reduced crop yields. Compared with manual weeding, intelligent robots can improve the efficiency of weeding operations. Developing real-time and reliable robotic systems for weed control in vegetable fields is a significant challenge due to the complexity of real-time identification, localization, and classification of vegetables as well as various weed species. The main purpose of this study was to propose a high-performance, lightweight deep learning model and an intra-row weed severity classification algorithm for real-time lettuce identification and weed severity classification. In this study, a scaling factor (τ = 0.5) was chosen to lightweight the YOLOv7 model. A new Multimodule-YOLOv7-L lightweight model was then developed by combining ECA and CA attention mechanisms, ELAN-B3 and DownC modules. The overall performance of the Multimodule-YOLOv7- L was better than that of other deep learning models, including YOLOv7, YOLOv7-Tiny, YOLOv8m, YOLOv5n-Cabbage, SE: YOLOv5x, YOLOv5s_Ghb, MST-YOLO_CBAM, Citrus-YOLOv7, Pineapple-YOLOv7, MS-YOLOv7 and CBAM-YOLOv7. The precision, recall, [email protected], F1-score, model weight and FPS of the Multimodule-YOLOv7- L model were 97.5 %, 95.7 %, 97.1 %, 96.6 %, 18.4 MB and 37.3 FPS (Image resolution about 3000 × 3000), respectively. An intra-row weed severity classification algorithm based on the Multimodule-YOLOv7-L model was proposed for use in a new mechanical-laser collaborative intra-row weeding robot. The developed algorithm achieved a classification accuracy of 100 % in eight lettuce weed scenarios, with the processing time of a single image ranging from 4-13 ms. The results of this study provided valuable reference for the development of intelligent robots for intra-row weed control. The algorithm proposed in this article can be obtained at https://github.com/H777R/The-intra-row-weed-severity-classification-algorithm.git.