Homography Based Player Identification in Live Sports

Yash Pandya, Kaustav Nandy, Shivam Agarwal; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 5209-5218

Modern live sports broadcasts display a wide variety of graphic visualizations identifying key players in a particular play. Traditionally, these graphics are created with extensive manual annotation for post-match analysis and take a significant amount of time to be produced. To create such visualizations in near real-time, automatic on-screen player identification and localization is essential. However, it is a challenging vision problem, especially for sports such as American football where the players wear elaborate protective equipment. In this work, we propose a novel approach which uses sensor data streams captured by wearables to automatically identify and locate on-screen players with low latency and high accuracy. The approach estimates a field registration homography using on-field player positions from RFID sensors, which is then used to identify and locate individual players on-screen. Experiments using American football data show that the method outperforms a deep learning based state-of-the-art(SOTA) vision-only field registration model both in terms of accuracy of the homography and also success rate of correct homography computation. On a dataset of over 150 replay clips, the proposed method correctly estimated the homography for approximately 25% additional clips as compared to the SOTA method. We demonstrate the efficacy of our method by applying it to the problem of rendering visualizations around key players within a few minutes of the live play. The player identification accuracy for these key players was over 96% across all clips, with an end-to-end latency of less than 1 minute.