Dynamic Re-Weighting for Long-Tailed Semi-Supervised Learning

Hanyu Peng, Weiguo Pian, Mingming Sun, Ping Li; Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023, pp. 6464-6474

The high demand for labeled data that characterizes deep learning is very labor-intensive. Semi-supervised Learning (SSL), acting as one of the breakthroughs, allows for the avoidance of this labeling loss thanks to its small amount of labeled data, alongside extracting information from a large amount of unlabeled data. And there is hope that the same performance for SSL can be achieved when compared to supervised learning methods. Regrettably, the research community has often developed SSL regarding the nature of a balanced data set; in contrast, real data is often imbalanced or even long-tailed. The need to study SSL under imbalance is therefore critical. In this paper, we shall essentially extend FixMatch (a SSL method) to the imbalanced case. We find that the unlabeled data is as well highly imbalanced during the training process; in this respect we propose a re-weighting solution based on the effective number. Furthermore, since prediction uncertainty leads to temporal variations in the number of pseudo-labels, we are innovative in proposing a dynamic re-weighting scheme on the unlabeled data. The simplicity and validity of our method are backed up by strong experimental evidence. Especially on CIFAR-10, CIFAR-100, ImageNet127 data sets, our approach provides the strongest results against previous methods across various scales of imbalance.