Referring Image Segmentation via Recurrent Refinement Networks

Ruiyu Li, Kaican Li, Yi-Chun Kuo, Michelle Shu, Xiaojuan Qi, Xiaoyong Shen, Jiaya Jia; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 5745-5753

Abstract


We address the problem of image segmentation from natural language descriptions. Existing deep learning-based methods encode image representations based on the output of the last convolutional layer. One general issue is that the resulting image representation lacks multi-scale semantics, which are key components in advanced segmentation systems. In this paper, we utilize the feature pyramids inherently existing in convolutional neural networks to capture the semantics at different scales. To produce suitable information flow through the path of feature hierarchy, we propose Recurrent Refinement Network (RRN) that takes pyramidal features as input to refine the segmentation mask progressively. Experimental results on four available datasets show that our approach outperforms multiple baselines and state-of-the-art.

Related Material


[pdf] [Supp]
[bibtex]
@InProceedings{Li_2018_CVPR,
author = {Li, Ruiyu and Li, Kaican and Kuo, Yi-Chun and Shu, Michelle and Qi, Xiaojuan and Shen, Xiaoyong and Jia, Jiaya},
title = {Referring Image Segmentation via Recurrent Refinement Networks},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}