MV-Soccer: Motion-Vector Augmented Instance Segmentation for Soccer Player Tracking

Fahad Majeed, Nauman Ullah Gilal, Khaled Al-Thelaya, Yin Yang, Marco Agus, Jens Schneider; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3245-3255

This work presents a novel real-time detection instance segmentation and tracking approach for soccer videos. Unlike conventional methods we augment video frames by incorporating motion vectors thus adding valuable shape cues that are not readily present in RGB frames. This facilitates improved foreground/background separation and enhances the ability to distinguish between players especially in scenarios involving partial occlusion. The proposed framework leverages the Cross-Stage-Partial Network53 (CSPDarknet53) as a backbone for instance segmentation and integrates motion vectors coupled with frame differencing. The model is simultaneously trained on two publicly available datasets and a private dataset SoccerPro which we created. The reason for simultaneous training is to reduce biases and increase generalization ability. To validate the effectiveness of our approach we conducted extensive experiments and attained 97% accuracy for the DFL - Bundesliga Data Shootout 98% on the SoccerNet-Tracking dataset and an impressive 99% on the SoccerPro (our) dataset.