Covariance-Based Space Regularization for Few-Shot Class Incremental Learning

Yijie Hu, Guanyu Yang, Zhaorui Tan, Xiaowei Wang, Kaizhu Huang, Qiu-Feng Wang; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 9548-9558

Few-shot Class Incremental Learning (FSCIL) presents a challenging yet realistic scenario which requires the model to continually learn new classes with limited labeled data (i.e. incremental sessions) while retaining knowledge of previously learned base classes (i.e. base sessions). Due to the limited data in incremental sessions models are prone to overfitting new classes and suffering catastrophic forgetting of base classes. To tackle these issues recent advancements resort to prototype-based approaches to constrain the base class distribution and learn discriminative representations of new classes. Despite the progress the limited data issue still induces ill-divided feature space leading the model to confuse the new class with old classes or fail to facilitate good separation among new classes. In this paper we aim to mitigate these issues by directly constraining the span of each class distribution from a covariance perspective. In detail we propose a simple yet effective covariance constraint loss to force the model to learn each class distribution with the same covariance matrix. In addition we propose a perturbation approach to perturb the few-shot training samples in the feature space which encourages the samples to be away from the weighted distribution of other classes. Regarding perturbed samples as new class data the classifier is forced to establish explicit boundaries between each new class and the existing ones. Our approach is easy to integrate into existing FSCIL approaches to boost performance. Experiments on three benchmarks validate the effectiveness of our approach achieving state-of-the-art performance.