Multi-Modal Aerial View Object Classification Challenge Results - PBVS 2023

Spencer Low, Oliver Nina, Angel D. Sappa, Erik Blasch, Nathan Inkawhich; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2023, pp. 412-421

This paper presents the findings and results of the third iteration of the Multi-modal Aerial View Object Classification (MAVOC) challenge in a detailed and comprehensive manner. The primary aim of both challenges is to encourage research into building recognition models that utilize both synthetic aperture radar (SAR) and electro-optical (EO) imagery. Participating teams are encouraged to develop multi-modal approaches that incorporate complementary information from both domains. While the 2021 challenge demonstrated the feasibility of combining both modalities, the 2022 challenge expanded on the capability of multi-modal models. The 2023 challenge introduces a refined version of the UNICORN dataset and demonstrates significant improvements made. The 2023 challenge adopts an updated UNIfied COincident Optical and Radar for recognitioN (UNICORN) dataset and competition format. Two tasks are featured: SAR classification and SAR + EO classification. In addition to measuring accuracy of models, we also introduce out-of-distribution measures to encourage model robustness. The majority of this paper is dedicated to discussing the top performing methods and evaluating their performance on our blind test set. It is worth noting that all of the top ten teams outperformed the Resnet-50 baseline. The top team for SAR classification achieved a 173% improvement over the baseline, while the top team for SAR + EO classification achieved a 175% improvement.