FisheyeBEVSeg: Surround View Fisheye Cameras based Bird's-Eye View Segmentation for Autonomous Driving

Senthil Yogamani, David Unger, Venkatraman Narayanan, Varun Ravi Kumar; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 1331-1334

Semantic segmentation is an effective way to perform scene understanding. Recently segmentation in 3D Bird's Eye View (BEV) space has become popular as its directly used by drive policy. However there is limited work on BEV segmentation for surround-view fisheye cameras commonly used in commercial vehicles. As this task has no real-world public dataset and existing synthetic datasets do not handle amodal regions due to occlusion we create a synthetic dataset using the Cognata simulator comprising diverse road types weather and lighting conditions. We generalize the BEV segmentation to work with any camera model; this is useful for mixing diverse cameras. We implement a baseline by applying cylindrical rectification on the fisheye images and using a standard LSS-based BEV segmentation model. We demonstrate that we can achieve better performance without undistortion which has the adverse effects of increased runtime due to pre-processing reduced field-of-view and resampling artifacts. Further we introduce a distortion-aware learnable BEV pooling strategy that is more effective for the fisheye cameras. We extend the model with an occlusion reasoning module which is critical for estimating in BEV space. Qualitative performance of FisheyeBEVSeg is showcased in the video at https://youtu.be/HfTPwMabgS0.