Hard Example Mining With Auxiliary Embeddings

Evgeny Smirnov, Aleksandr Melnikov, Andrei Oleinik, Elizaveta Ivanova, Ilya Kalinovskiy, Eugene Luckyanets; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 37-46

Hard example mining is an important part of the deep embedding learning. Most methods perform it at the mini-batch level. However, in the large-scale settings there is only a small chance that proper examples will appear in the same mini-batch and will be coupled into the hard example pairs or triplets. Doppelganger mining was previously proposed to increase this chance by means of class-wise similarity. This method ensures that examples of similar classes are sampled into the same mini-batch together. One of the drawbacks of this method is that it operates only at the class level, while there also might be a way to select appropriate examples within class in a more elaborated way than randomly. In this paper, we propose to use auxiliary embeddings for hard example mining. These embeddings are constructed in such way that similar examples have close embeddings in the cosine similarity sense. With the help of these embeddings it is possible to select new examples for the mini-batch based on their similarity with the already selected examples. We propose several ways to create auxiliary embeddings and use them to increase the number of potentially hard positive and negative examples in each mini-batch. Our experiments on the challenging Disguised Faces in the Wild (DFW) dataset show that hard example mining with auxiliary embeddings improves the discriminative power of learned representations.