RLNet: Robust Linearized Networks for Efficient Private Inference

Sreetama Sarkar, Souvik Kundu, Peter A. Beerel; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 244-253

The growing concern about data privacy has led to the development of private inference (PI) frameworks in client-server applications which protects both data privacy and model IP. However the cryptographic primitives required yield significant latency overheads which limits their widespread application. At the same time changing environments demand PI services to be robust against various naturally occurring and gradient-based perturbations. Despite several works focused on the development of latency-efficient models suitable for PI the impact of these models on robustness has remained unexplored. Towards this goal this paper presents RLNet a class of models that can yield latency improvement via the reduction of high-latency ReLU operations while improving the model performance on both clean and corrupted images. In particular RLNet models provide a "triple win ticket" of improved classification accuracy on clean naturally perturbed and gradient-based perturbed images using a shared-mask shared-weight architecture with over an order of magnitude fewer ReLUs than baseline models. To demonstrate the efficacy of RLNet we perform extensive experiments with ResNet and WRN model variants on CIFAR-10 CIFAR-100 and Tiny-ImageNet datasets. Our experimental evaluations show that RLNet can yield models with up to 11.14x fewer ReLUs with accuracy close to the all-ReLU models on clean naturally perturbed and gradient-based perturbed images. Compared with the SoTA non-robust linearized models at similar ReLU budgets RLNet achieves an improvement in adversarial accuracy of up to 47% in naturally perturbed accuracy of up to 16.4% while improving clean image accuracy up to 1.5%.