Hot-Started NAS for Task-Specific Embedded Applications

Lotte Hendrickx, Wiebe Van Ranst, Toon Goedemé; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 1971-1978

Neural architecture search (NAS) has proven its worth in discovering new neural networks. Combining the possibility to satisfy multiple objectives in one search, it is especially useful for getting the most out of embedded devices with limited resources. However, research into small and efficient neural networks precedes NAS. We investigate the influence of combining this pre-existing knowledge with NAS techniques, for which we propose to hot-start the NAS search with a human-designed optimal network. Our experiments show that doing so speeds up the NAS process significantly, but the resulting optimal model at the end is only marginally better. Since embedded devices are often used for a specific task, we also explore the impact of using a task-specific dataset in the NAS process. Our experiments demonstrate that for a constrained problem, a smaller network can be found as compared to a general problem.