Kiwifruit segmentation and identification of picking point on its stem in orchards

Abstract

Automated picking of kiwifruit with retained stems is crucial for extending the fruit’s freshness period and ensuring its quality during storage. Accurately obtaining kiwifruit picking points based on kiwifruit stem detection is necessary to effectively achieve this goal. The small size and similar colour characteristics of kiwifruit stems to fruit make fruit stem detection more difficult and pose a challenge in accurately identifying picking points. This study proposed a DS-UNet method based on improved convolutional networks as a biomedical image segmentation model for the segmentation of kiwifruit and its stem, identification of picking points to segment the characteristics of kiwifruit and its stems and identification and localisation of the corresponding picking points in trellis cultivation. First, to improve convolutional networks for biomedical image segmentation (UNet) models, conventional convolution is replaced by depth-wise-separable convolution in the encoding stage. A spatial attention mechanism is added after the convolutional layer in the decoding stage, which increases the model’s computing power and segmentation efficiency. Then, constraint conditions were set to establish the relationship between the fruit stem and fruit and lock the target fruit stem by determining the positional relationship between the growth of the kiwifruit and its stems. Finally, the centroid of the minimum bounding rectangle of the kiwifruit stem characteristic area was identified and used as an effective target for fruit stem picking point. Experimental results demonstrate that the proposed DS-UNet instance segmentation algorithm can achieve increased mPA, mIoU, P and R values for kiwifruit and its stems by 6.76%, 10.98%, 10.10% and 12.46%, respectively, compared to those of the original UNet. The inference time was shortened by 87.50%. Using the proposed method, the probability of effectively predicting the picking point was 91.65%. This study provides a solid foundation for developing an information perception system for smart picking equipment and the storage and fresh-keeping of kiwifruit after harvest. This study also provides a reference for picking point prediction of other fruits and vegetables with similar growth characteristics.