Diversified and Multi-Class Controllable Industrial Defect Synthesis for Data Augmentation and Transfer

Jing Wei, Fei Shen, Chengkan Lv, Zhengtao Zhang, Feng Zhang, Huabin Yang; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 4445-4453

Data augmentation is crucial to solve few-sample issues in industrial inspection based on deep learning. However, current industrial data augmentation methods have not yet demonstrated on-par ability in the synthesis of complex defects with pixel-level annotations. This paper proposes a new defect synthesis framework to fill the gap. Firstly, DCDGANc (Diversified and multi-class Controllable Defect Generation Adversarial Networks based on constant source images) is proposed to employ class labels to construct source inputs to control the category and random codes to generate diversified styles of defects. DCDGANc can generate defect content images with pure backgrounds, which avoids the influence of non-defect information and makes it easy to obtain binary masks by segmentation. Secondly, the Poisson blending is improved to avoid content loss when blending generated defect contents to the normal backgrounds. Finally, the complete defect samples and accurate pixel-level annotations are obtained by fine image processing. Experiments are conducted to verify the effectiveness of our work in wood, fabric, metal, and marble. The results show that our methods yield significant improvement in the segmentation performance of industrial products. Moreover, our work enables zero-shot inspection by facilitating defect transfer between datasets with different backgrounds but similar defects, which can greatly reduce the cost of data collection in industrial inspection.