A Reality Check on Pre-training for Exemplar-free Class-Incremental Learning

Eva Feillet, Adrian Popescu, Céline Hudelot; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7614-7625

Abstract


Exemplar-free class-incremental learning (EFCIL) aims to classify streaming data without storing examples from the past. Recent EFCIL works suggest that (i) models pre-trained with large amounts of data should be used to initialize learning (ii) self-supervised learned transformers generalize better than supervised convolutional models (iii) adding generated data to the pre-training dataset can improve incremental accuracy. In this article we question the above assertions by comprehensively evaluating various initial training strategies combined with four EFCIL algorithms using four large-scale datasets. Our results indicate that: (i) pre-trained models are preferable when the domain of the incremental classification task is well represented in the pre-training datasets but training with initial data remains useful when the domain shift is significant (ii) supervised convolutional networks remain competitive particularly when improving representation transferability using data augmentation or a projector (iii) adding classes from an external dataset to train the initial model boosts performance when the initial set of classes is small but has a limited effect otherwise (iv) additional classes generated with a diffusion model are not necessarily more useful than a well-chosen set of ImageNet classes to improve model transferability. We provide a nuanced analysis of these results and formulate recommendations to facilitate the practical adoption of EFCIL algorithms.

Related Material


[pdf] [supp]
[bibtex]
@InProceedings{Feillet_2025_WACV, author = {Feillet, Eva and Popescu, Adrian and Hudelot, C\'eline}, title = {A Reality Check on Pre-training for Exemplar-free Class-Incremental Learning}, booktitle = {Proceedings of the Winter Conference on Applications of Computer Vision (WACV)}, month = {February}, year = {2025}, pages = {7614-7625} }