Improving Rare Classes on nuScenes LiDAR Segmentation Through Targeted Domain Adaptation

Vickram Rajendran, Chuck Tang, Frits van Paasschen; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 130-139

We generate synthetic data in order to target improvement on specific rare classes in LiDAR segmentation without regressing performance on the existing, plentiful classes. While using auxiliary data to improve performance on a domain is not new with respect to classification, there is limited research on targeting specific classes with this technique. It is currently unclear how to extend those methods to work well on more complicated but realistic use cases for autonomous driving such as LiDAR segmentation. By upsampling specific classes in the auxiliary domain, mixing data between domains, and splitting representation building and fine-tuning, we are able to see impressive improvements on a targeted rare class without losing performance on the other classes. On the popular autonomous driving benchmark nuScenes, we use this procedure to improve performance on the rare class of cyclists by 18%, resulting in the best Cylinder3D model on the LiDAR segmentation benchmark. We also show that these techniques extend to other classes (debris) and other tasks (LiDAR object detection), giving strong evidence that this methodology generalizes well to other autonomous perception tasks.