-
Cross-modal Embeddings for Video and Audio Retrieval
AbstractIn this work, we explore the multi-modal information provided by the Youtube-8M dataset by projecting the audio and visual features into a common feature space, to obtain joint audio-visual embeddings. These links are used to retrieve audio samples that fit well to a given silent video, and also to retrieve images that match a given query audio. The results in terms of Recall@K obtained over a subset of YouTube-8M videos show the potential of this unsupervised approach for cross-modal feature learning.
Related Material
[pdf] [arXiv][bibtex]@InProceedings{Suris_2018_ECCV_Workshops,
author = {Suris, Didac and Duarte, Amanda and Salvador, Amaia and Torres, Jordi and Giro-i-Nieto, Xavier},
title = {Cross-modal Embeddings for Video and Audio Retrieval},
booktitle = {Proceedings of the European Conference on Computer Vision (ECCV) Workshops},
month = {September},
year = {2018}
}
The ECCV 2018 workshop papers, provided here by the Computer Vision Foundation, are the author-created versions. The content of the papers is identical to the content of the officially published ECCV 2018 LNCS version of the papers as available on SpringerLink: https://link.springer.com/conference/eccv.
This material is presented to ensure timely dissemination of scholarly and technical work.
Copyright and all rights therein are retained by authors or by other copyright holders.
All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright.