Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning

Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, Matthew R. Scott; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 5022-5030

Abstract


A family of loss functions built on pair-based computation have been proposed in the literature which provide a myriad of solutions for deep metric learning. In this pa-per, we provide a general weighting framework for under-standing recent pair-based loss functions. Our contributions are three-fold: (1) we establish a General Pair Weighting (GPW) framework, which casts the sampling problem of deep metric learning into a unified view of pair weighting through gradient analysis, providing a powerful tool for understanding recent pair-based loss functions; (2) we show that with GPW, various existing pair-based methods can be compared and discussed comprehensively, with clear differences and key limitations identified; (3) we propose a new loss called multi-similarity loss (MS loss) under the GPW,which is implemented in two iterative steps (i.e., mining and weighting). This allows it to fully consider three similarities for pair weighting, providing a more principled approach for collecting and weighting informative pairs. Finally, the proposed MS loss obtains new state-of-the-art performance on four image retrieval benchmarks, where it outperforms the most recent approaches, such as ABE[14] and HTL[4], by a large margin, e.g.,60.6%-65.7%on CUB200,and 80.9%-88.0%on In-Shop Clothes Retrieval datasetat Recall@1.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Wang_2019_CVPR,
author = {Wang, Xun and Han, Xintong and Huang, Weilin and Dong, Dengke and Scott, Matthew R.},
title = {Multi-Similarity Loss With General Pair Weighting for Deep Metric Learning},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}