Revisiting Self-Supervised Visual Representation Learning

Alexander Kolesnikov, Xiaohua Zhai, Lucas Beyer; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 1920-1929

Abstract


Unsupervised visual representation learning remains a largely unsolved problem in computer vision research. Among a big body of recently proposed approaches for unsupervised learning of visual representations, a class of self-supervised techniques achieves superior performance on many challenging benchmarks. A large number of the pretext tasks for self-supervised learning have been studied, but other important aspects, such as the choice of convolutional neural networks (CNN), has not received equal attention. Therefore, we revisit numerous previously proposed self-supervised models, conduct a thorough large scale study and, as a result, uncover multiple crucial insights. We challenge a number of common practices in self-supervised visual representation learning and observe that standard recipes for CNN design do not always translate to self-supervised representation learning. As part of our study, we drastically boost the performance of previously proposed techniques and outperform previously published state-of-the-art results by a large margin. We will release the code for reproducing our experiments when the anonymity requirements are lifted.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Kolesnikov_2019_CVPR,
author = {Kolesnikov, Alexander and Zhai, Xiaohua and Beyer, Lucas},
title = {Revisiting Self-Supervised Visual Representation Learning},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2019}
}