Auto-Fit: A Human-Machine Collaboration Feature for Fitting Bounding Box Annotations

Abstract

Large high-quality annotated datasets are essential in training deep learning models, but are expensive and time-consuming to create. A large chunk of time in the annotation process goes into adjusting bounding boxes to fit the desired object. In this paper, we propose the facilitation of human machine collaboration through the creation of an Auto-Fit feature which automatically tightens an initial bounding box around an object being annotated. The challenge lies in making this feature class agnostic in order to allow its usage regardless of the type of object being annotated. This is achieved through the use of various computer vision algorithms to extract the desired object as a foreground mask, determine the coordinates of its extremities, and redraw the bounding box based on these new coordinates. The best results were achieved with the Grabcut algorithm, which attained an accuracy of 84.69% on small boxes. The Pytorch implementation of ResNet-101 pre-trained on the COCO train2017 dataset is also used as a foreground extractor in one iteration of the implementation, in order to provide a baseline comparison between the performance of a computer vision-based solution versus one based on a standalone object detection model. This garnered an accuracy of 83.04% on small boxes, showing that the computer vision-based solution is able to surpass the accuracy of a standalone object detection model.