An Empirical Study of Language CNN for Image Captioning

Jiuxiang Gu, Gang Wang, Jianfei Cai, Tsuhan Chen; The IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1222-1231

Abstract


Language models based on recurrent neural networks have dominated recent image caption generation tasks. In this paper, we introduce a Language CNN model which is suitable for statistical language modeling tasks and shows competitive performance in image captioning. In contrast to previous models which predict next word based on one previous word and hidden state, our language CNN is fed with all the previous words and can model the long-range dependencies in history words, which are critical for image captioning. The effectiveness of our approach is validated on two datasets: Flickr30K and MS COCO. Our extensive experimental results show that our method outperforms the vanilla recurrent neural network based language models and is competitive with the state-of-the-art methods.

Related Material


[pdf] [arXiv]
[bibtex]
@InProceedings{Gu_2017_ICCV,
author = {Gu, Jiuxiang and Wang, Gang and Cai, Jianfei and Chen, Tsuhan},
title = {An Empirical Study of Language CNN for Image Captioning},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
month = {Oct},
year = {2017}
}