Multi-Capture Dynamic Calibration of Multi-Camera Systems

Avinash Kumar, Manjula Gururaj, Kalpana Seshadrinathan, Ramkumar Narayanswamy; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2018, pp. 1843-1851

Multi-camera systems have seen an emergence in various consumer devices enabling many applications e.g. bokeh (Apple IPhone), 3D measurement (Dell Venue 8) etc. An accurately calibrated multi-camera system is essential for proper functioning of these applications. Usually, a onetime factory calibration with technical targets is done to accurately calibrate such systems. Although accurate, factory calibration does not hold over the life time of the device as normal wear and tear, thermal effects, device usage etc. can cause calibration parameters to change. Thus, a dynamic or self-calibration based on multi-view image features is required to refine calibration parameters. One of the important factors governing the accuracy of dynamic calibration is the number and distribution of feature points in the captured scene. A dense feature distribution enables better sampling of the 3D scene, while avoiding degenerate situations (e.g. all features on one plane), thus sufficiently modeling the forward imaging process for calibration. But, single real life images with dense feature distribution are difficult or nearly impossible to capture e.g. texture-less indoor or occluded scenes. In this paper, we propose a new multi-capture paradigm for multi-camera dynamic calibration where multiple multi-view images of different 3D scenes (thus varying feature point distribution) are jointly used to calibrate the multi-camera system. We present a new optimality criteria to select the best set of candidate images from a pool of multi-view images, along with their order, to use for multi-capture dynamic calibration. We also propose a methodology to jointly model calibration parameters of multiple multi-view images. Finally, we show improved performance of multi-capture dynamic calibration over single-capture dynamic calibration in terms of lower epipolar rectification and 3D measurement error.