Wildlife Image Generation From Scene Graphs

Yoshio Rubio, Marco A. Contreras-Cruz; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 305-314

Image generation from natural language descriptions is an exciting and challenging task in computer vision and natural language processing. In this work, we propose a novel method to generate synthetic images from scene graphs in the context of wildlife scenarios. Given a scene graph, our method uses a graph convolutional network to predict semantic layouts, and a semi-parametric approach based on a cascade refinement network to synthesize the final image. We test our approach on a subset of COCO dataset, which we call COCO-Wildlife. Our results outperform the baselines, both quantitatively and qualitatively, and the visual results show the ability of our approach to generate stunning images with natural interaction between the different objects. Our findings show the potential to expand the use case of the proposed method to other contexts where scale and realism is fundamental.