Exploring the Limits: Applying State-of-the-Art Stereo Matching Algorithms to Rectified Ultra-Wide Stereo

Filip Slezák, Morten S. Laursen, Thomas B. Moeslund; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 1335-1344

Stereo cameras leveraging two-view geometry have predominantly focused on narrow field-of-view rectified stereo using the pinhole camera model. This research trend overlooks the complexities and potential of wide-angle stereo systems which necessitate the use of wide-angle fisheye optics that can not be well approximated by pinhole camera model. Consequently a lack of standardized form leads researchers to explore various strategies. Currently a dichotomy exists between utilizing raw images directly or rectifying them. Wide-angle stereo rectification opens the potential to reuse the latest state-of-the-art (SOTA) algorithms designed for pinhole rectified stereo as a black box. However rectification comes at the cost of severe distortions throughout the image and non-linear triangulation of 3D structure. The literature currently lacks a thorough examination of the implications of these distortions and the impact of applying the latest SOTA algorithms to stereo-rectified wide-angle images. Our work addresses this gap by conducting an exhaustive analysis of the wide-angle rectified stereo framework delivering concrete recommendations for developing accurate wide-angle stereo systems.