Must Unsupervised Continual Learning Relies on Previous Information?

Haoyang Cheng, Haitao Wen, Heqian Qiu, Lanxiao Wang, Minjian Zhang, Hongliang Li; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 5519-5529

Open-world recognition has recently gained significant attention owing to its ability to bridge the gap between experimental scenarios and real-world applications. Since continual learning can learn from a sequence of dynamic data streams it obtains extensive applications in open-world recognition. However because of the production of data annotation is usually time-consuming and labor-intensive in real-world scenarios it's necessary to develop unsupervised continual learning. Recent studies start to investigate unsupervised continual learning (i.e. UCL) but mainly focus on rehearsal and regularization strategies to enhance the anti-forgetting capability of UCL.In practice rehearsal and regularization are information-dependent which require information from previous data as supervised signals e.g. replayed data and previous model. In this paper we propose an information-free method Alternate Task Discrimination (ATD) which is a self-supervised pretext task for continuity and improves anti-forgetting capability via encouraging the model to discriminate which data stream current sample is from. The whole process doesn't rely on any previous information. In order to perform ATD effectively in UCL framework we design an alternating optimization algorithm where UCL and ATD are optimized respectively. We validate the effectiveness of the proposed method on multiple standard UCL benchmarks where it obtains considerable improvements compared with baseline methods. In addition our approach can be used as a plug-in unit which makes further achievements when collaborated with existing popular UCL methods.