Test-time Specialization of Dynamic Neural Networks

Sam Leroux, Dewant Katare, Aaron Yi Ding, Pieter Simoens; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 1048-1056

In recent years there has been a notable increase in the size of commonly used image classification models. This growth has empowered models to recognize thousands of diverse object types. However their computational demands pose significant challenges especially when deploying them on resource-constrained edge devices. In many use cases where a model is deployed on an edge device only a small subset of the classes will ever be observed by a given model instance. Our proposed test-time specialization of dynamic neural networks allows these models to become faster at recognizing the classes that are observed frequently while maintaining the ability to recognize all other classes albeit slightly less efficient. We benchmark our approach on a real-world edge device obtaining significant speedups compared to the baseline model without test-time adaptation.