Point Proposal based Instance Segmentation with Rectangular Masks for Robot Picking Task

Satoshi Ito, Susumu Kubota; Proceedings of the Asian Conference on Computer Vision (ACCV), 2020

In this paper, we focus on instance segmentation of a top-view image for robot picking task. One difficulty in this setting is that objects are located in various orientations and highly overlapped, where a traditional box proposal approach such as Mask R-CNN does not work well because more than one objects often have very similar bounding-boxes. To address this issue, we adopt a recently developed point proposal approach. This approach firstly generates point proposals instead of box proposals, then an instance mask is predicted over an image for each proposal point. This procedure enables us to obtain pixel-precise masks even for objects sharing the same bounding-box. However, mask prediction over an image may produce a few false positive pixels apart from objects and these false positives are problematic for robot picking task. To suppress them, we introduce rectangular masks. A rectangular mask for each proposal point restricts the existence area of the corresponding object within the rectangle. The experimental result on WISDOM dataset shows that our method achieves superior performance to Mask R-CNN with the same backbone model and introduction of rectangular masks gives small improvement of mask AP and large improvement of box AP.