BiMAE - A Bimodal Masked Autoencoder Architecture for Single-Label Hyperspectral Image Classification

Maksim Kukushkin, Martin Bogdan, Thomas Schmid; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 2987-2996

Hyperspectral imaging offers manifold opportunities for applications that may not or only partially be achieved within the visual spectrum. Our paper presents a novel approach for Single-Label Hyperspectral Image Classification demonstrated through the example of a key challenge faced by agricultural seed producers: seed purity testing. We employ Self-Supervised Learning and Masked Image Modeling techniques to tackle this task. Recognizing the challenges and costs associated with acquiring hyperspectral data we aim to develop a versatile method capable of working with visible arbitrary combinations of spectral bands (multispectral data) and hyperspectral sensor data. By integrating RGB and hyperspectral data we leverage the detailed spatial information from RGB images and the rich spectral information from hyperspectral data to enhance the accuracy of seed classification. Through evaluations in various real-life scenarios we demonstrate the flexibility scalability and efficiency of our approach.