Evaluation of segment anything model (SAM) for automated labelling in machine learning classification of UAV geospatial data

Abstract

With the present trend toward digitization in many areas of urban planning and development, accurate object classification is becoming increasingly vital. To develop machine learning models that can effectively classify the broader region, it is crucial to have accurately labelled datasets for object extraction. However, the process of generating sufficient labelled data for machine learning models remains challenging. A recently developed AI-assisted segmentation approach called the Segment Anything Model (SAM) offers a solution to enhance the labelling of complex and intricate image structures. By utilizing SAM, the accuracy and consistency of annotation results can be improved, while also significantly reducing the time required for annotation. This paper aims to assess the efficiency of SAM annotated labels for training machine learning models using high-resolution remote sensing data captured by UAVs (Unmanned Aerial Vehicles) in the peri-urban region of Anad, Kerala, India. A comparative analysis was conducted to evaluate the performance of training datasets generated using SAM and manual labelling with existing tools. Multiple machine learning models, including Random Forest, Support Vector Machine, and XGBoost, were employed for this analysis. The findings demonstrate that employing the XGBoost algorithm in combination with SAM annotated labels yielded an accuracy of 78%. In contrast, the same algorithm trained with the manually labeled dataset achieved an accuracy of only 68%. A similar pattern was observed when employing the Random Forest algorithm, with accuracies of 78% and 60% while using SAM annotated labels and manual labels, respectively. These outcomes unequivocally showcase the enhanced effectiveness and dependability of the SAM-based segmentation method in producing accurate results.