Segmentation of Plant-part from 3D Point Cloud Using Deep Learning and Multi-view Vision

Introduction: Automatic measuring and monitoring technologies make great revolution to modern agriculture in the 21st century. High-throughput, precision, and non-destructive plant phenotyping measurement has become a key issue in agricultural eld. Researchers obtain 3D structure information of plant by reconstructing 3D models of plant such as point cloud [5, 6]. Point cloud is a kind of data that can represent 3D depth information. In recent years, the application of deep learning methods on images has achieved amazing results. However, the application of deep learning methods on point cloud is still potential for exploration. The existing deep learning methods for point cloud data [4] are mainly developed from two aspects. One is non-point-based methods , such as multi-view-based [9], and voxel-based [12]. The other is pointbased method, such as PointNet [8] network, which provides an end-to-end learning method that can directly process point cloud. It is strong, but it does not have the ability to capture local information, so PointNet++ [7] was proposed for iterative feature extraction through the eld of each point so that the network can better extract the local features of the point cloud. In plant phenotyping research, we want to automatically obtain plant phenotypic parameters through three-dimensional point clouds. One of the key steps is to segment the point clouds of di erent scenes by an instance. Then we can obtain the number, position and size of the plants in the scene. We noticed that the 3D-BoNet [11] point cloud instance segmentation network has the characteristics of simple design, universal and e cient. The above network characteristics are worthy that we research deep learning application for the acquisition of plant phenotypic parameters based on this framework. In this study, we consider two segmentation tasks from 3D point cloud: