SkiLL: Skipping Color and Label Landscape: Self Supervised Design Representations for Products in E-Commerce

Vinay K. Verma, Dween Rabius Sanny, Shreyas Sunil Kulkarni, Prateek Sircar, Abhishek Singh, Deepak Gupta; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 3503-3507

Understanding the design of a product without human supervision is a crucial task for e-commerce services. Such a capability can help in multiple downstream e-commerce tasks like product recommendations, design trend analysis, image-based search, and visual information retrieval, etc. For this task, getting fine-grain label data is costly and not scalable for the e-commerce product. In this paper, we leverage knowledge distillation based self-supervised learning (SSL) approach to learn design representations. These representations do not require human annotation for training and focus on only design related attributes of a product and ignore attributes like color, orientation, etc. We propose a global and task specific local augmentation space which captures the desired image information and provides robust visual embedding. We evaluated our model for the three highly diverse datasets, and also propose and measure a quantitative metric to evaluate the model's color invariant feature learning ability. In all scenarios, our proposed approach outperforms the recent SSL model by upto 8.6% in terms of accuracy.