SoccerNet Game State Reconstruction: End-to-End Athlete Tracking and Identification on a Minimap

Vladimir Somers, Victor Joos, Anthony Cioppa, Silvio Giancola, Seyed Abolfazl Ghasemzadeh, Floriane Magera, Baptiste Standaert, Amir M. Mansourian, Xin Zhou, Shohreh Kasaei, Bernard Ghanem, Alexandre Alahi, Marc Van Droogenbroeck, Christophe De Vleeschouwer; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3293-3305

Tracking and identifying athletes on the pitch holds a central role in collecting essential insights from the game such as estimating the total distance covered by players or understanding team tactics. This tracking and identification process is crucial for reconstructing the game state defined by the athletes' positions and identities on a 2D top-view of the pitch (i.e. a minimap). However reconstructing the game state from videos captured by a single camera is challenging. It requires understanding the position of the athletes and the viewpoint of the camera to localize and identify players within the field. In this work we formalize the task of Game State Reconstruction and introduce SoccerNet-GSR a novel Game State Reconstruction dataset focusing on football videos. SoccerNet-GSR is composed of 200 video sequences of 30 seconds annotated with 9.37 million line points for pitch localization and camera calibration as well as over 2.36 million athlete positions on the pitch with their respective role team and jersey number. Furthermore we introduce GS-HOTA a novel metric to evaluate game state reconstruction methods. Finally we propose and release an end-to-end baseline for game state reconstruction bootstrapping the research on this task. Our experiments show that GSR is a challenging novel task which opens the field for future research. Our dataset and codebase are publicly available at https://github.com/SoccerNet/sn-gamestate.