When Visual State Space Model Meets Backdoor Attacks

Sankalp Nagaonkar, Achyut Mani Tripathi, Ashish Mishra; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7419-7428

The recently proposed Visual State Space Model (VMamba) operating on the principle of state space mechanisms (SSM) processes images as a sequence of patches and outperforms Vision Transformers (ViT) in several computer vision tasks. Given their substantial design differences from CNNs and ViT it is crucial to investigate their vulnerability to backdoor attacks and the impact of various advanced backdoor attacks on their robustness. Backdoor attacks involve embedding a specific trigger into a small subset of training images which remains dormant until activated later. While the model performs well on clean test images an attacker can manipulate its decisions by presenting the trigger in one of the test images. This work examines state-of-the-art (SOTA) vision architectures (ResNet ViT MLP-mixer VMamba) focusing on their susceptibility to backdoor attacks and the effect of different backdoor attacks on their robustness. The well-known Visual State Space Model (VMamba) is the least susceptible to backdoor attacks among these architectures. To address this in this paper we propose two novel QR decomposition-based backdoor attacks that are visually imperceptible and achieve a high attack success rate (ASR) against the VMamba. We also present a qualitative analysis of the proposed backdoor attacks explaining the reasons behind their success or failure against the VMamba model. Experiments and results conducted on two popular image datasets (CIFAR-10 and ImageNet-1K) demonstrate that the proposed backdoor attacks exceed the performance of SOTA backdoor attacks and effectively fool the recently proposed VMamba model.