Practical framework for generative on-branch soybean pod detection in occlusion and class imbalance scenes

Abstract

The number of pods per plant can serve as an effective indicator of soybean yield, and accurately determining this is essential for evaluating high-quality soybean varieties. However, traditional manual pod counting is time-consuming and laborious. Although deep learning-based pod detection methods have attracted much attention, there are still considerable challenges for the effective detection of pods in occlusion and class imbalance scenes. As a remedy, this study proposes a framework that leverages synthetic pod image generation and multi-stage transfer learning to generate detection model of on-branch soybean pods in complex scenes. This framework employs a novel pipeline: initially separating individual pods from non-occluded pod images in an off-branch pod training set, then using these to generate synthetic datasets with diverse pod features. Next, a multi-stage transfer learning method is employed to train an on-branch pod detection model, leveraging both real and synthetic datasets to enhance pod feature extraction in complex scenes. The detection model of proposed framework, YOLOv7-tiny (tiny version of You Only Look Once v7), integrates an angle prediction module based on Circular Smooth Label for rotated object detection, Coordinate Attention modules for enhanced feature extraction and Minimum Point Distance Intersection over Union Loss for precise bounding box perception. Experimental results show that proposed framework achieves an 81.1% mAP (mean Average Precision) for detecting on-branch pods in complex scenes, surpassing the best-performing model by 23.7%. This proposed method presents an effective solution for complex on-branch pod detection, having great potential of serving as robust pipeline for similar agricultural tasks.