Human-machine collaboration on data annotation of images by semi-automatic labeling

Deployment of deep neural network architectures in computer vision applications requires labeled images which human workers create in a manual, cumbersome process of drawing bounding boxes and segmentation masks. In this work, we propose an image labeling companion that supports human workers to label images faster and more efficiently. Our data-pipeline utilizes One-Shot, Few-Shot and pre-trained object detection models to provide bounding box suggestions, thereby reducing the required user interactions during labeling to corrective adjustments. The resulting labels are then used to continuously update the underlying suggestion models. Optionally, we apply a refinement step, where an available bounding box is converted into a finer segmentation mask. We evaluate our approach with a group of participants who label images using our tool - both manually and with the system. In all our experiments, the achieved quality is consistently comparable with manually created labels at factor 2 to 6 faster execution times.