Uncertainty-based Forgetting Mitigation for Generalized Few-Shot Object Detection

Karim Guirguis, George Eskandar, Mingyang Wang, Matthias Kayser, Eduardo Monari, Bin Yang, Jürgen Beyerer; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 2586-2595

Generalized Few-Shot Object Detection (G-FSOD) seeks to jointly detect base classes with abundant data and novel classes with limited data. Due to data scarcity predictive uncertainties are more pronounced in G-FSOD than in conventional object detection. Unaccounting for these uncertainties leads to degraded overall detection performance and forgetting the base classes. However previous G-FSOD works have not exploited these uncertainties. Upon examining the basic two-stage G-FSOD framework which includes a Region Proposal Network (RPN) and a subsequent R-CNN we observe that a straightforward integration of uncertainty estimation leads to detrimental performance. To this end we first increase the model capacity by increasing the depth of the RPN and cascading multiple R-CNNs in an end-to-end manner. Next we interleave the stages with uncertainty estimation and attention blocks. The aim is to progressively refine the proposals by exploiting the estimated uncertainties while attending to the discriminative features through the attention mechanism. Extensive experiments on the well-established G-FSOD benchmarks MS-COCO and PASCAL-VOC show that our proposed method sets a new G-FSOD standard.