Dense object detection based canopy characteristics encoding for precise spraying in peach orchards

Abstract

Accurate and precise spraying in orchards is paramount for optimized agricultural practices, ensuring efficient pesticide utilization, minimized environmental impact, and enhanced crop yield by targeting specific areas with the right amount of treatment. The asymmetrical distribution of foliage and flowers in peach orchards poses a formidable challenge to achieving precise spray accuracy, impeding the uniform application of treatments and compromising the overall efficacy of pest and disease control measures. In response to the prevailing challenges in achieving accurate spray application caused by the asymmetrical distribution of foliage and flowers in peach orchards, this paper introduces a novel deep neural network to map the RGB image and corresponding depth to the density map of peach flowers or foliage. The model consists of components: (1) two backbones based on ResNet-50 that extract contextual features from the RGB image and depth features from depth data at multiple scales and levels; (2) an optimized depth-enhanced module that effectively fuses the distinct features extracted from the two input streams; and (3) a two-stage decoder that aggregates the high-level cross-modal features to regress the coarse density map and subsequently integrates it with the low-level cross-modal features for final density map prediction. To evaluate the performance of our model, we collected 493 frames (206,095 instances) of peach flowers and 475 frames (350,833 instances) of foliage from the peach orchards utilizing our sprayer prototype equipped with stereo cameras. The proposed method outperforms state-of-the-art models on our datasets, demonstrating the superiority and efficacy for encoding canopy characteristics in the form of flower and foliage density maps for blossom and cover sprays. It attains significant computational efficiency, exhibiting a frame rate of 20 FPS, and showcases exceptional accuracy with a WMAPE of 12.11% for peach flowers and a WMAPE of 13.37% for leaves.