Leaf only SAM: A segment anything pipeline for zero-shot automated leaf segmentation

Abstract

Segment Anything Model (SAM) is a new “foundation model” that can be used as a zero-shot object segmentation method with the use of either guide prompts such as bounding boxes, polygons, or points. Alternatively, additional post processing steps can be used to identify objects of interest after segmenting everything in an image. Here a method is presented using segment anything together with a series of post processing steps to segment potato leaves, called Leaf Only SAM. The advantage of this proposed method is that it does not require any training data to produce its results so has many applications across the field of plant phenotyping where there is limited high quality annotated data available. The performance of Leaf Only SAM is compared to a Mask R-CNN model which has been fine-tuned on a small novel potato leaf dataset. On the evaluation dataset, Leaf Only SAM finds an average recall of 73.1 and an average precision of 73.9, compared to recall of 87.6 and precision of 84.4 for Mask R-CNN. Leaf Only SAM does not perform better than the fine-tuned Mask R-CNN model on the potato leaf dataset, but the SAM based model does not require any extra training or annotation. This shows there is potential to use SAM as a zero-shot classifier with the addition of post processing steps.