Efficient Hardware-aware Neural Architecture Search for Image Super-resolution on Mobile Devices

Xindong Zhang, Hui Zeng, Lei Zhang; Proceedings of the Asian Conference on Computer Vision (ACCV), 2022, pp. 721-738

With the ubiquitous use of mobile devices in our daily life, how to design a lightweight network for high-performance image super-resolution (SR) has become increasingly important. However, it is difficult and laborious to manually design and deploy different SR models on different mobile devices, while the existing network architecture search (NAS) techniques are expensive and unfriendly to find the desired SR networks for various hardware platforms. To mitigate these issues, we propose an efficient hardware-aware neural architecture search (EHANAS) method for SR on mobile devices. First, EHANAS supports searching in a large network architecture space, including the macro topology (e.g., number of blocks) and microstructure (e.g., kernel type, channel dimension, and activation type) of the network. By introducing a spatial and channel masking strategy and a re-parameterization technique, we are able to finish the whole searching procedure using one single GPU card within one day. Second, the hardware latency is taken as a direct constraint on the searching process, enabling hardware-adaptive optimization of the searched SR model. Experiments on two typical mobile devices demonstrate the effectiveness of the proposed EHANAS method, where the searched SR models obtain better performance than previously manually designed and automatically searched models. The source codes of EHANAS can be found at https://github.com/xindongzhang/EHANAS.