Boosting Instance Segmentation With Synthetic Data: A Study To Overcome the Limits of Real World Data Sets

Florentin Poucin, Andrea Kraus, Martin Simon; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 945-953

A major issue related to computer vision for the automotive industry is that real-world perception models require huge amount of well-annotated data to achieve decent performance. While this data is very expensive to collect and annotate, synthetically generated images seem to be an efficient alternative to solve this problem. More and more public data sets, composed of synthetic data, are available in various domains, however, there is too little concrete methodology to use them properly. In this paper, we propose a simple approach combining the use of synthetic and real images to boost instance segmentation. We mention some pre-processing requirements as harmonizing instance labeling and removing non-valuable instances from synthetic images. We present our training strategy based on data set mixing, and show that it overcomes the domain shift between real and synthetic data sets. A comparison study with other training approaches, such as fine-tuning techniques, highlights the benefits of our method, which boosts network performances on both real and synthetic image inferences.