Knowing When to Look For What and Where: Evaluating Generation of Spatial Descriptions with Adaptive Attention

Mehdi Ghanimifard, Simon Dobnik; Proceedings of the European Conference on Computer Vision (ECCV) Workshops, 2018, pp. 0-0

Abstract


We examine and evaluate adaptive attention [17] (which balances the focus on visual features and focus on textual features) in generating image captions in end-to-end neural networks, in particular how adaptive attention is informative for generating spatial relations. We show that the model generates spatial relations more on the basis of textual rather than visual features and therefore confirm the previous observations that the learned visual features are missing information about geometric relations between objects.

Related Material


[pdf]
[bibtex]
@InProceedings{Ghanimifard_2018_ECCV_Workshops,
author = {Ghanimifard, Mehdi and Dobnik, Simon},
title = {Knowing When to Look For What and Where: Evaluating Generation of Spatial Descriptions with Adaptive Attention},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},
month = {September},
year = {2018}
}