Semi-supervised multi-class tree crown delineation using aerial multispectral imagery and lidar data

The segmentation of individual trees based on deep learning is more accurate than conventional meth- ods. However, a sufficient amount of training data is mandatory to leverage the accuracy potential of deep learning-based approaches. Semi-supervised learning techniques, by contrast, can help simplify the time-consuming labelling process. In this study, we introduce a new semi-supervised tree segmen- tation approach for the precise delineation and classification of individual trees that takes advantage of pre-clustered tree training labels. Specifically, the instance segmentation Mask R-CNN is combined with the normalized cut clustering method, which is applied to lidar point clouds. The study areas were located in the Bavarian Forest National Park, southeast Germany, where the tree composition includes coniferous, deciduous and mixed forest. Important tree species are European beech (Fagus sylvatica), Norway spruce (Picea abies) and silver fir (Abies alba). Multispectral image data with a ground sample distance of 10 cm and laser scanning data with a point density of approximately 55 points/m² were acquired in June 2017. From the laser scanning data, three-channel images with a resolution of 10 cm were generated. The models were tested in seven reference plots in the national park, with a total of 516 trees measured on the ground. When the color infrared images were used, the experiments demonstrated that the Mask R-CNN models, trained with the tree labels generated through lidar-based clustering, yielded mean F1 scores of 79 % that were up to 18 % higher than those of the normalized cut baseline method and thus significantly improved. Similarly, the mean over- all accuracy of the classification results for the coniferous, deciduous, and standing deadwood tree groups was 96 % and enhanced by up to 6 % compared with the baseline classification approach. The experiments with lidar-based images yielded slightly worse (1–2 %) results both for segmentation and for classification. Our study demonstrates the utility of this simplified training data preparation pro- cedure, which leads to models trained with significantly larger amounts of data than is feasible with with manual labelling. The accuracy improvement of up to 18 % in terms of the F1 score is further evidence of its advantages.