Can Relevance Feedback, Conversational Search and Foundation Models Work Together for Interactive Video Search and Exploration?

Ujjwal Sharma, Omar Shahbaz Khan, Stevan Rudinac, Björn Þór Jónsson; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2025, pp. 3749-3758

Abstract

Exquisitor is an interactive system that supports search and exploration in large multimedia collections by integrating conversational search with relevance feedback (RF). However, combining these approaches introduces challenges, including reconciling user expectations with system capabilities, mitigating over-reliance on text-based queries when RF may be more effective, and bridging feedback modalities across conversational and RF paradigms. This work proposes extensions to Exquisitor that leverage foundation models for query expansion, reformulation and refinement. By transparently adjusting user-submitted text queries in real-time, these extensions aim to enhance search effectiveness and improve the overall user experience.

Related Material

[pdf]
[bibtex]
@InProceedings{Sharma_2025_CVPR, author = {Sharma, Ujjwal and Khan, Omar Shahbaz and Rudinac, Stevan and J\'onsson, Bj\"orn {\TH}\'or}, title = {Can Relevance Feedback, Conversational Search and Foundation Models Work Together for Interactive Video Search and Exploration?}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2025}, pages = {3749-3758} }