Scalable and Explainable Outfit Generation

Alexander Lorbert, David Neiman, Arik Poznanski, Eduard Oks, Larry Davis; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2021, pp. 3931-3934

Abstract


We present an end-to-end system for learning outfit recommendations. The core problem we address is how a customer can receive clothing/accessory recommendations based on a current outfit and what type of item the customer wishes to add to the outfit. Using a repository of coherent and stylish outfits, we leverage self-attention to learn a mapping from the current outfit and the customer-requested category to a visual descriptor output. This output is then fed into nearest-neighbor-based visual search, which, during training, is learned via triplet loss and mini-batch retrievals. At inference time, we use a beam search with a desired outfit composition to generate outfits at scale. Moreover, the attention networks provide a diagnostic look into the recommendation process, serving as a fashion-based sanity check.

Related Material


[pdf]
[bibtex]
@InProceedings{Lorbert_2021_CVPR, author = {Lorbert, Alexander and Neiman, David and Poznanski, Arik and Oks, Eduard and Davis, Larry}, title = {Scalable and Explainable Outfit Generation}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2021}, pages = {3931-3934} }