Boundary-Aware Instance Segmentation

Zeeshan Hayder, Xuming He, Mathieu Salzmann; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 5696-5704

We address the problem of instance-level semantic seg- mentation, which aims at jointly detecting, segmenting and classifying every individual object in an image. In this con- text, existing methods typically propose candidate objects, usually as bounding boxes, and directly predict a binary mask within each such proposal. As a consequence, they cannot recover from errors in the object candidate genera- tion process, such as too small or shifted boxes. In this paper, we introduce a novel object segment rep- resentation based on the distance transform of the object masks. We then design an object mask network (OMN) with a new residual-deconvolution architecture that infers such a representation and decodes it into the final binary object mask. This allows us to predict masks that go beyond the scope of the bounding boxes and are thus robust to inaccu- rate object candidates. We integrate our OMN into a Mul- titask Network Cascade framework, and learn the result- ing boundary-aware instance segmentation (BAIS) network in an end-to-end manner. Our experiments on the PAS- CAL VOC 2012 and the Cityscapes datasets demonstrate the benefits of our approach, which outperforms the state- of-the-art in both object proposal generation and instance segmentation.