Towards Explainable Visual Vessel Recognition Using Fine-Grained Classification and Image Retrieval

Heiko Karus, Friedhelm Schwenker, Michael Munz, Michael Teutsch; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 82-92

The precise recognition of vessel types is critical for applications in maritime surveillance but manual visual inspection is slow and error-prone. Automated fine-grained object recognition helps to quickly and accurately categorize vessels as long as they overcome challenges of modern deep and machine learning-based methods such as class imbalance or intransparency. This paper utilizes recent literature to work towards this task considering two different problem formulations: fine-grained classification and image retrieval. We create a novel dataset called Military MARVEL consisting of 15858 images of 137 military vessel classes to conduct the extensive survey-like experiments. On two maritime datasets we demonstrate that end-to-end fine-grained classification leads to slightly higher accuracy for the price of less flexibility compared to image retrieval. Heatmap-based eXplainable Artificial Intelligence methods are combined with the vessel recognition approaches to assess the achieved transparency quantitatively and qualitatively. Notably quantitative measures are consistent with qualitative evaluation especially in fine-grained classification. Our Military MARVEL dataset is available at https://github.com/HensoldtOptronicsCV/FineGrainedVesselRecognition.