GRASP-GCN: Graph-Shape Prioritization for Neural Architecture Search Under Distribution Shifts

Sofia Casarin, Oswald Lanz, Sergio Escalera; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 1695-1703

Neural Architecture Search (NAS) methods have shown to output networks that largely outperform human-designed networks. However conventional NAS methods have mostly tackled the single dataset scenario incurring in a large computational cost as the procedure has to be run from scratch for every new dataset. In this paper we focus on predictor-based algorithms and propose a simple and efficient way of improving their prediction performance when dealing with data distribution shifts. We exploit the Kronecker-product on the randomly wired search-space and create a small NAS benchmark composed of networks trained over four different datasets. To improve the generalization abilities we propose GRASP-GCN a ranking Graph Convolutional Network which takes as additional input the shape of the layers of the neural networks. GRASP-GCN is trained with the not-at-convergence accuracies and improves the state-of-the-art of 3.3 % for Cifar-10 increasing moreover the generalization abilities under data distribution shift.