SegBuilder: A Semi-Automatic Annotation Tool for Segmentation

Md Alimoor Reza, Eric Manley, Sean Chen, Sameer Chaudhary, Jacob Elafros; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 8483-8492

This paper addresses the problem of image annotation for segmentation tasks. Semantic segmentation involves labeling each pixel in an image with predefined categories such as sky cars roads and humans. Deep learning models require numerous annotated images for effective training but manual annotation is slow and time-consuming. To mitigate this challenge we leverage the Segment Anything Model (SAM)- a vision foundation model. We introduce SegBuilder a framework that incorporates SAM to automatically generate segments which are then tagged by human annotators using a quick selection list. To demonstrate SegBuilder's effectiveness we introduced a novel dataset for image segmentation in underwater environments featuring animals such as sea lions beavers and jellyfish. Experiments on this dataset showed that SegBuilder significantly speeds up the annotation process compared to the publicly available tool Label Studio. SegBuilder also includes a free-form drawing tool allowing users to create correct segments missed by SAM. This feature is particularly useful for scenes with shadows camouflaged objects and part-based segmentation tasks where SAM falls short. Experimentally we demonstrated SegBuilder's efficacy in these scenarios showcasing its potential for generating pixel-wise annotations crucial for training robust deep learning models for semantic segmentation.