Few-Shot Structured Domain Adaptation for Virtual-to-Real Scene Parsing

Junyi Zhang, Ziliang Chen, Junying Huang, Liang Lin, Dongyu Zhang; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019, pp. 0-0

A structured domain adaptation (SDA) model for virtual-to-real scene parsing, learning to predict visual structure labels in real-world target scenes via mitigating the statistical discrepancy between large scale labeled virtual source and unlabeled real-world target images. But different from the source images drawn from urban simulation platforms, the target images could be expansive and difficult to collect at scale in real-world scenes. Besides, the trend of urbanization constantly changes the visual appearances of target scenes, which encourages SDA models to quickly adapt to new target scenes by merely given very few target images for training. To address the concerns, we attempt to achieve the virtual-to-real scene parsing from a new perspective inspired by few-shot learning. Instead of using a large amount of unlabeled target data used in existing SDA models, our few-shot SDA model takes a few of target real images with semantic labels in each scene, which collaborates with virtual source domain to train a virtual-to-real scene parser. Specifically, our framework is a two-stage adversarial network which contains a scene parser and two discriminators. Based on the data pairing method, our framework can handle the problem of scarce target data well and make full use of the limited semantic labels. We evaluate our method on two suites of virtual-to-real scene parsing setups. The experimental results show that our method exceeds the state-of-the-art SDA model by 7.1% in mIoU on SYNTHIA-to-CITYSCAPES and 4.03% in mIoU on GTA5-to-CITYSCAPES in the case of 1-shot.