Multi-Modal Large Language Model with RAG Strategies in Soccer Commentary Generation

Xiang Li, Yangfan He, Shuaishuai Zu, Zhengyang Li, Tianyu Shi, Yiting Xie, Kevin Zhang; Proceedings of the Winter Conference on Applications of Computer Vision (WACV), 2025, pp. 6197-6206

As a globally celebrated sport soccer has seen its appeal greatly amplified by engaging and vivid commentary. Recently Multi-Modal Large Language Models (MLLMs) have attracted attention in generating soccer commentaries due to their remarkable capacities of understanding different modalities of the input videos. Most of these methods have shown that the use of multiple modalities can enhance the commentary quality which includes video audio and structured meta-data. However delivering precise and rich commentary requires the ability to accurately discern subtle differences in similar backgrounds events and players. This presents a significant challenge for existing MLLMs. So we propose SoccerComment a framework for generating soccer commentary that integrates MLLMs with Retrieval-Augmented Generation (RAG) strategies. This framework enhances inference efficiency and reduces the need for continuous training through a multi-modal clustering memory unit and retrieval-augmented in-context learning mechanisms ultimately improving the accuracy and diversity of the commentary. Based on similar retrieved scenarios SoccerComment demonstrates outstanding zero-shot performance offering a new direction and scalable solution for future research in soccer commentary generation.