Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis

Chuan Li, Michael Wand; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2479-2486

Abstract


This paper studies a combination of generative Markov random field (MRF) models and discriminatively trained deep convolutional neural networks (dCNNs) for synthesizing 2D images. The generative MRF acts on higher-levels of a dCNN feature pyramid, controling the image layout at an abstract level. We apply the method to both photographic and non-photo-realistic (artwork) synthesis tasks. The MRF regularizer prevents over-excitation artifacts and reduces implausible feature mixtures common to previous dCNN inversion approaches, permitting synthezing photographic content with increased visual plausibility. Unlike standard MRF-based texture synthesis, the combined system can both match and adapt local features with considerable variability, yielding results far out of reach of classic generative MRF methods.

Related Material


[pdf]
[bibtex]
@InProceedings{Li_2016_CVPR,
author = {Li, Chuan and Wand, Michael},
title = {Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2016}
}