-
[pdf]
[supp]
[arXiv]
[bibtex]@InProceedings{Kalluri_2024_CVPR, author = {Kalluri, Tarun and Wang, Weiyao and Wang, Heng and Chandraker, Manmohan and Torresani, Lorenzo and Tran, Du}, title = {Open-world Instance Segmentation: Top-down Learning with Bottom-up Supervision}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2024}, pages = {2693-2703} }
Open-world Instance Segmentation: Top-down Learning with Bottom-up Supervision
Abstract
Top-down instance segmentation architectures excel with predefined closed-world taxonomies but exhibit biases and performance degradation in open-world scenarios. In this work we introduce bottom-Up and top-Down Open-world Segmentation (UDOS) a novel approach that combines classical bottom-up segmentation methods within a top-down learning framework. UDOS leverages a top-down network trained with weak supervision derived from class-agnostic bottom-up segmentation to predict object parts. These part-masks undergo affinity-based grouping and refinement to generate precise instance-level segmentations. UDOS balances the efficiency of top-down architectures with the capacity to handle unseen categories through bottom-up supervision. We validate UDOS on challenging datasets (MS-COCO LVIS ADE20k UVO and OpenImages) achieving superior performance over state-of-the-art methods in cross-category and cross-dataset transfer tasks. Our code and models will be publicly available.
Related Material