Misalignment-Free Relation Aggregation for Multi-Source-Free Domain Adaptation

Hao-Wei Yeh, Qier Meng, Tatsuya Harada; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2023, pp. 4313-4322

In multi-source-free domain adaptation (MSFDA), it is important to effectively fuse latent features from multiple source models to improve adaptation performance on target domain. Existing works weightedly sum source-model features for fusion, which cannot fully leverage the discriminativity of features due to misaligned semantics, and is not applicable to source models with non-identical feature dimensionalities. To mitigate these issues, we propose the idea of misalignment-free relation aggregation (MFRA): instead of directly summing the features, we aggregate the similarity relationships between target samples in each source-model feature space. Specifically, for each source model, we first compute the similarities between the target sample of interest and all the other target samples. The resulting similarities are then summed along the source models to produce the aggregated similarity. To leverage the aggregated similarity in adaptation, a peer-supervised contrastive learning and an adversarial training scheme are designed to transfer discriminative information among models. The method not only effectively preserves discriminativity from each source model after summation, but also is applicable to source models with non-identical feature dimensionalities. The proposed method achieves accuracies higher or comparable to existing MSFDA methods on various cross-domain object recognition tasks. Further studies are also conducted to verify the effectiveness of aggregating inter-sample relationships, as well as the applicability of proposed method under non-identical source-model feature dimensionalities.