Annotating Object Instances With a Polygon-RNN

Lluis Castrejon, Kaustav Kundu, Raquel Urtasun, Sanja Fidler; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 5230-5238

In this paper, we propose an approach for semi-automatic annotation of object instances. While most current methods treat object segmentation as a pixel-labeling problem, we here cast it as a polygon prediction task, mimicking how most current datasets have been annotated. In particular, our approach takes as input an image crop and produces a vertex of the polygon, one at a time, allowing the human annotator to interfere at any time and correct the point. Our model easily integrates any correction, producing as accurate segmentations as desired by the annotator. We show that our annotation method speeds up the annotation process by factor of 4.7 across all classes, while achieving 78.4% agreement in IoU with original ground-truth, matching the typical agreement between human annotators. For cars, our speed-up factor is even higher, at 7.3 for agreement of 82.2%. We further show generalization capabilities of our approach on unseen datasets.