Analysis of adapter in attention of change detection Vision Transformer

Ryunosuke Hamada, Tsubasa Minematsu, Cheng Tang, Atsushi Shimada; Proceedings of the Asian Conference on Computer Vision (ACCV) Workshops, 2024, pp. 34-49

Vision Transformer (ViT) contributes to accurate change detection with robustness to background changes. However, retraining ViT requires a large amount of computation to adapt to unlearned scenes. This study investigates the addition of learnable parameters into change detection ViT to reduce the computational complexity of retraining. We introduce MLP as an adapter as an addition to the attention output and the residual connection of the change detection ViT and apply LoRA method to the change detection ViT. We evaluate the retraining of additional parameter models for various background changes and analyze proper setting of additional parameters to adapt the target scenes. Introducing MLP and LoRA to change detection ViT improves the accuracy for the target scenes without competition between two additional parameter methods.