Learning With Average Precision: Training Image Retrieval With a Listwise Loss

Jerome Revaud, Jon Almazan, Rafael S. Rezende, Cesar Roberto de Souza; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 5107-5116

Abstract


Image retrieval can be formulated as a ranking problem where the goal is to order database images by decreasing similarity to the query. Recent deep models for image retrieval have outperformed traditional methods by leveraging ranking-tailored loss functions, but important theoretical and practical problems remain. First, rather than directly optimizing the global ranking, they minimize an upper-bound on the essential loss, which does not necessarily result in an optimal mean average precision (mAP). Second, these methods require significant engineering efforts to work well, e.g., special pre-training and hard-negative mining. In this paper we propose instead to directly optimize the global mAP by leveraging recent advances in listwise loss formulations. Using a histogram binning approximation, the AP can be differentiated and thus employed to end-to-end learning. Compared to existing losses, the proposed method considers thousands of images simultaneously at each iteration and eliminates the need for ad hoc tricks. It also establishes a new state of the art on many standard retrieval benchmarks. Models and evaluation scripts have been made available at: https://europe.naverlabs.com/Deep-Image-Retrieval/.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Revaud_2019_ICCV,
author = {Revaud, Jerome and Almazan, Jon and Rezende, Rafael S. and Souza, Cesar Roberto de},
title = {Learning With Average Precision: Training Image Retrieval With a Listwise Loss},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2019}
}