Development of plant phenotyping system using Pan Tilt Zoom camera and verification of its validity

Abstract

Quantitative analysis for plant growth attributes has gained prominence in plant science and agriculture. Despite the availability of automated phenotyping systems as a solution to labor-intensive manual measurement techniques, these systems often require specialized knowledge and face challenges in scaling for high-throughput applications. This research introduces a scalable high-throughput plant phenotyping technique utilizing a Pan Tilt Zoom (PTZ) camera. The primary objective is to assess the application of a PTZ camera in a plant phenotyping system. By integrating open-source software and hardware technologies, the method captures images of cucumber plants in a controlled greenhouse environment. The operational procedure of the robot consists of a series of steps. It begins with the robot’s initial movement to capture infrared images, followed by an analysis to detect Aruco markers serving as location identifiers for capturing plant images. Subsequently, the PTZ camera is adjusted to capture specific plant traits from predefined viewpoints. The captured images with location IDs, preset viewpoints, and timestamps are then sent to a remote server. Validation of the system’s dependability includes manual measurements on fundamental operations and the evaluation of the effectiveness of zoomed images captured by the PTZ camera, tested through plant feature detection. Experimental results demonstrate promising outcomes, achieving a mean average precision (mAP) of 94%, 97.6%, 98.4%, 90.1%, and 97.6% for apical buds, male flowers, female flowers, tiny cucumbers, and mature cucumbers respectively when using the trained YOLOv8s on an augmented dataset tested on highly zoomed images validation set, outperforming less zoomed or less detailed validation sets. These findings underscore the efficacy of this innovative approach in capturing real-time plant images. Leveraging the PTZ camera’s zoom, pan, and tilt capabilities enables comprehensive visualization of plant traits and adaptability to evolving growth patterns, thereby improving the results of plant feature detection. The amassed imagery serves a dual purpose by acting as training data for AI models, highlighting their potential to facilitate future research endeavors demanding extensive and scalable plant information.