Learning Network Architectures of Deep CNNs Under Resource Constraints

Michael Chan, Daniel Scarafoni, Ronald Duarte, Jason Thornton, Luke Skelly; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 1703-1710

Abstract


Recent works in deep learning have been driven broadly by the desire to attain high accuracy on certain challenge problems. The network architecture and other hyper-parameters of many published models are typically chosen by trial-and-error experiments with little considerations paid to resource constraints at deployment time. We propose a fully automated model learning approach that (1) treats architecture selection as part of the learning process, (2) uses a blend of broad-based random sampling and adaptive iterative refinement to explore the solution space, (3) performs optimization subject to given memory and computational constraints imposed by target deployment scenarios, and (4) is scalable and can use only a practically small number of GPUs for training. We present results that show graceful model degradation under strict resource constraints for object classification problems using CIFAR-10 in our experiments. We also discuss future work in further extending the approach.

Related Material


[pdf]
[bibtex]
@InProceedings{Chan_2018_CVPR_Workshops,
author = {Chan, Michael and Scarafoni, Daniel and Duarte, Ronald and Thornton, Jason and Skelly, Luke},
title = {Learning Network Architectures of Deep CNNs Under Resource Constraints},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
month = {June},
year = {2018}
}