Scattering Prompt Tuning: A Fine-tuned Foundation Model for SAR Object Recognition

Weilong Guo, Shengyang Li, Jian Yang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3056-3065

Synthetic Aperture Radar (SAR) serves as a vital tool in various earth observation applications providing robust imaging under challenging weather conditions. While the fine-tuned foundation models excel in many downstream tasks they struggle with SAR object recognition because of SAR's unique imaging and scattering characteristics. In this study we propose a novel approach named Scattering Prompt Tuning (SPT) based vision foundation model. It uses SAR image scattering information as a prompt and integrates learnable parameters into the pre-trained model's input space to help learn SAR's unique information. We also employ a lightweight Residual AdapterMLP for fine-tuning design a Sequential Feature Aggregation (SFA) to selectively fuse features from different transformer blocks effectively and develop a Dynamic Distributional Contrast loss (DCLoss) to maintain the proper distance between different objects in feature space. Additionally a four-stage training strategy incorporating semi-supervised learning is deployed to enhance SAR object recognition performance further. Our approach reaches a Top-1 accuracy of 37.9% and an AUROC of 0.83 on the final dataset winning the first place in the SAR Classification track of PBVS 2024 Multi-modal Aerial View Object Classification Challenge which is better than the latest advanced fine-tuned foundation models.