A Universal Protocol to Benchmark Camera Calibration for Sports

Floriane Magera, Thomas Hoyoux, Olivier Barnich, Marc Van Droogenbroeck; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 3335-3346

Camera calibration is a crucial component in the realm of sports analytics as it serves as the foundation to extract 3D information out of the broadcast images. Despite the significance of camera calibration research in sports analytics progress is impeded by outdated benchmarking criteria. Indeed the annotation data and evaluation metrics provided by most currently available benchmarks strongly favor and incite the development of sports field registration methods i.e. methods estimating homographies that map the sports field plane to the image plane. However such homography-based methods are doomed to overlook the broader capabilities of camera calibration in bridging the 3D world to the image. In particular real-world non-planar sports field elements (such as goals corner flags baskets ...) and image distortion caused by broadcast camera lenses are out of the scope of sports field registration methods.To overcome these limitations we designed a new benchmarking protocol named ProCC based on two principles: (1) the protocol should be agnostic to the camera model chosen for a camera calibration method and (2) the protocol should fairly evaluate camera calibration methods using the reprojection of arbitrary yet accurately known 3D objects. Indirectly we also provide insights into the metric used in SoccerNet-calibration which solely relies on image annotation data of viewed 3D objects as ground truth thus implementing our protocol. With experiments on the World Cup 2014 CARWC and SoccerNet datasets we show that our benchmarking protocol provides fairer evaluations of camera calibration methods. By defining our requirements for proper benchmarking we hope to pave the way for a new stage in camera calibration for sports applications with high accuracy standards.