Text Query Based Traffic Video Event Retrieval With Global-Local Fusion Embedding

Thang-Long Nguyen-Ho, Minh-Khoi Pham, Tien-Phat Nguyen, Hai-Dang Nguyen, Minh N. Do, Tam V. Nguyen, Minh-Triet Tran; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2022, pp. 3134-3141

Abstract


Retrieving event videos based on textual description is a promising research topic in the fast-growing data field. However, traffic data increases every day, so it is essential to need intelligent traffic system management in conjunction with humans to speed up the search. We propose a multi-module system that delivers accurate results that meet objectives, including explainability and scalability at the same time. Our solution considers neighbors entities related to the mentioned object to represent an event by rule-based, which can represent an event by the relationship of multiple objects. We also propose to add a modified model from last year's Alibaba model with an explainable architecture. As the traffic data is vehicle-centric, we apply two language and image modules to analyze the input data and obtain the global properties of the context and the internal attributes of the vehicle. We introduce a one-on-one dual training strategy for each representation vector to optimize the interior features for the query. Finally, a refinement module gathers previous results to enhance the final retrieval result. We benchmarked our approach on the data of the AI City Challenge 2022 and got the best results at an MMR of 0.3611. We were ranked in the top 4 on 50% of the test set and in the top 5 on the full dataset.

Related Material


[pdf]
[bibtex]
@InProceedings{Nguyen-Ho_2022_CVPR, author = {Nguyen-Ho, Thang-Long and Pham, Minh-Khoi and Nguyen, Tien-Phat and Nguyen, Hai-Dang and Do, Minh N. and Nguyen, Tam V. and Tran, Minh-Triet}, title = {Text Query Based Traffic Video Event Retrieval With Global-Local Fusion Embedding}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2022}, pages = {3134-3141} }