A robust plant localization and identification system for precision farming – The effects of uncontrolled illumination on an end-to-end deep-learning plant detection algorithm

In this research, we investigated the robustness of an end-to-end deep-learning plant detection algorithm with respect to influences from uncontrolled illumination. For this research, we acquired two datasets, one containing images of plants taken under controlled illumination and one dataset with images acquired under uncontrolled illumination. We trained and evaluated the YOLOv3 object detector on both datasets. The object detector scored a mean Average Precision of 0.96 on controlled illumination conditions and 0.90 on the uncontrolled illumination conditions. This difference in performance is significant. Introduction The development of environmental awareness has led to an increased demand for efficient use of nutrients, agrochemicals, and other resources in agriculture. Nutrients and agrochemicals are most efficiently used when applied plant specifically. For plant-specific precision farming, a vision-based plant localization and identification system(hereafter plantdetection system) is a key enabling technology. In the last decade, extensive research on the development of plantdetection systems is performed. However, the developed plant-detection systems cannot deal with the variation preFig. 1: Prototype of a volunteer potato spraying robot. The camera position is indicated by the red circle. VDI-Berichte Nr. 2361, 2019 383 https://doi.org/10.51202/9783181023617-383 Generiert durch IP '54.70.40.11', am 22.05.2021, 18:38:26. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. sent in the agricultural environment. These systems lack robustness to uncontrolled natural illumination [1], different soil types [2][3], variation in growth stages [4] and occluding plant parts [5]. In this study, we used recently developed deep-learning computer-vision algorithms. These algorithms have demonstrated their robustness and generalization properties in a wide range of tasks (e.g. obstacle detection and scene understanding for autonomous vehicles). Despite the promising results achieved in other domains, no deep-learning algorithm is reported with a plant-detection rate above 95% [6][7][8][9]. Furthermore, most detection systems require a covered setup with artificial lighting, imposing constraints on the applicability and capacity. Within this paper, we investigated the difference in plant detection performance of the YOLOv3 object detector on images acquired in controlled illumination conditions versus images acquired under uncontrolled illuminated conditions. As use case, a small scale volunteer potato spraying robot, as shown in Fig. 1 was used. Materials and methods Dataset description On the 28th of May and the 1st of June in 2018, data was acquired at the Wageningen University farm in Valthermond in The Netherlands, shown in Fig. 2. During the data campaign, two datasets were acquired: The controlled light, and the uncontrolled light dataset. The controlled light dataset contains images of sugar beets and volunteer potatoes Fig. 2: Location of the fields used for the datasets. VDI-Berichte Nr. 2361, 2019 384 https://doi.org/10.51202/9783181023617-383 Generiert durch IP '54.70.40.11', am 22.05.2021, 18:38:26. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. acquired under controlled illumination. To mitigate the effects of ambient sunlight, a tractormounted cover was used. Underneath this cover, homogeneous lighting was assured by an LED ceiling, shown in Fig. 3. The uncontrolled light dataset contains images acquired under uncontrolled natural illumination by using the prototype robot shown in Fig. 1. The dataset is acquired on the 28th of May and the 1st of June between 10:00-11:00 and 15:00-16:00 in direct sunlight with no clouds. Both datasets are acquired with an IDS uEye CP camera with a resolution of 2076 x 2076 pixels. The camera was equipped with a Kowa LM5JC10M lens at maximum aperture for maximum light capture. Furthermore, the camera’s software was conFig.d to use automatic white-balance and automatic gain. In both datasets, 526 images are annotated for bounding box detection of the sugar beet and volunteer potato plants. Image samples from the two datasets are given in Fig. 4. Fig. 4: The left image comes from the controlled light dataset. The right image comes from Fig. 3: In the left image, the setup used to block the ambient light is shown. Underneath the setup, homogeneous lighting is assured by a LED ceiling, shown in the right image. VDI-Berichte Nr. 2361, 2019 385 https://doi.org/10.51202/9783181023617-383 Generiert durch IP '54.70.40.11', am 22.05.2021, 18:38:26. Das Erstellen und Weitergeben von Kopien dieses PDFs ist nicht zulässig. the uncontrolled light dataset. Note that the image from the controlled light dataset is homogeneously illuminated and has no shadows. Contrary, the image from the uncontrolled light dataset contains hard shadows and bright sunlight. Detection algorithm Considering the use case of an automated volunteer potato removal robot, a lightweight algorithm that can run in real-time on a mobile pc was needed. Therefore, the YOLOv3 object detector developed by [11] and implemented by [12] was used. This object detector is one of the fastest object detectors available, and almost as accurate as the state of the art object detectors such as Faster RCNN and Retinanet [11]. The YOLOv3 algorithm is trained on images rescaled to a resolution of 416x416 pixels. The model was initialized with the pre-trained weights provided by [13], and trained for 500 epochs using default settings, listed in Appendix A. The training images were subjected to data augmentation on the image geometry and image color, using the parameters given in