Knockoff Branch: Model Stealing Attack via Adding Neurons in the Pre-Trained Model

Li-Ying Hung, Cooper Cheng-Yuan Ku; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 7062-7070

We introduce Knockoff Branch: adding few neurons as a knockoff container for learning stolen features. Model stealing attacks extract the functionality from the victim model by querying APIs. Prior work substantially enhanced transferability and improved query efficiency between the adversary model and the victim model. However there is still a limited understanding of the knockoff itself. For knockoff the model is either compared to the same type but with different structures or different types and capacities. For this reason we propose a framework to analyze the knockoff quality for a single model specifically reinvestigating transformer-based extraction. We observed that 1) when the adversary can access the public pretrained model full fine-tuning is not necessary. This allows a knockoff to require only about 0.5% of trainable parameters and 20 epochs. 2) Although querying by out-of-distribution datasets leads to a sub-optimal knockoff this issue can be mitigated by scaling branch features even without using complicated sampling strategies. Our proposed method is lightweight and achieves high accuracy at most similar to white-box knowledge distillation (higher performance than the victim model). https://github.com/onlyin-hung/knockoff-branch.