Towards Learning Image Similarity from General Triplet Labels

Radu Dondera; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 1181-1190

Metric learning for images has so far focused overwhelmingly on a class-based definition of similarity: two images are similar if they belong to the same class and dissimilar otherwise. Impressive results were achieved on datasets for fine grained categorization but performance is nearing saturation. Recent work in neuroscience and psychology produced datasets with other types of similarity labels e.g. the outlier in a group of three but traditional metric learning methods are ill-suited to such data because of low density of labels. To overcome this difficulty we propose a novel approach in the teacher-student learning paradigm. Multiple teacher models learn to embed images based only on relations with other images and then a student model learns to embed images based on both content and dense relations provided by the teachers. We show significant improvement over existing triplet based metric learning methods both in result quality and in training efficiency. Additionally through experiments on class based datasets we show the generality of approaching metric learning via knowledge transfer. Code is available at github.com/greenfieldvision/tdml .