Active Transferability Estimation

Tarun Ram Menta, Surgan Jandial, Akash Patil, Saketh Bachu, Vimal K.B, Balaji Krishnamurthy, Vineeth N. Balasubramanian, Mausoom Sarkar, Chirag Agarwal; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 2659-2670

As transfer learning techniques are increasingly used to transfer knowledge from the source model to the target task it becomes important to quantify which source models are suitable for a given target task without performing computationally expensive fine-tuning. Inspired by active learning techniques we propose ACT (ACtive Transferability) a new strategy to improve the performance of transferability estimation methods by leveraging an informative subset of the target data. By leveraging the model's internal and output representations we introduce two techniques - class-agnostic and class-aware - to identify informative subsets and show that ACT can be applied to any existing transferability metric to improve their performance and reliability. Our experimental results across multiple source model architectures target datasets and transfer learning tasks show that ACT metrics are consistently better or on par with the state-of-the-art transferability metrics.