On-Demand Learning for Deep Image Restoration

Ruohan Gao, Kristen Grauman; Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1086-1095

While machine learning approaches to image restoration offer great promise, current methods risk training models fixated on performing well only for image corruption of a particular level of difficulty--such as a certain level of noise or blur. First, we examine the weakness of conventional "fixated" models and demonstrate that training general models to handle arbitrary levels of corruption is indeed non-trivial. Then, we propose an on-demand learning algorithm for training image restoration models with deep convolutional neural networks. The main idea is to exploit a feedback mechanism to self-generate training instances where they are needed most, thereby learning models that can generalize across difficulty levels. On four restoration tasks--image inpainting, pixel interpolation, image deblurring, and image denoising--and three diverse datasets, our approach consistently outperforms both the status quo training procedure and curriculum learning alternatives.