End-To-End Instance Segmentation With Recurrent Attention

Mengye Ren, Richard S. Zemel; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 6656-6664

Abstract


While convolutional neural networks have gained impressive success recently in solving structured prediction problems such as semantic segmentation, it remains a challenge to differentiate individual object instances in the scene. Instance segmentation is very important in a variety of applications, such as autonomous driving, image captioning, and visual question answering. Techniques that combine large graphical models with low-level vision have been proposed to address this problem; however, we propose an end-to-end recurrent neural network (RNN) architecture with an attention mechanism to model a human-like counting process, and produce detailed instance segmentations. The network is jointly trained to sequentially produce regions of interest as well as a dominant object segmentation within each region. The proposed model achieves competitive results on the CVPPP, KITTI, and Cityscapes datasets.

Related Material


[pdf] [video]
[bibtex]
@InProceedings{Ren_2017_CVPR,
author = {Ren, Mengye and Zemel, Richard S.},
title = {End-To-End Instance Segmentation With Recurrent Attention},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {July},
year = {2017}
}